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Overview 


Research  completed  since  the  1980s  has  yielded  substantial  knowledge  about  how  to  help  welfare  recipients 
and  other  low-income  individuals  prepare  for  and  find  jobs.  Many  participants  in  these  successful  job  prepara- 
tion and  placement  programs,  however,  ended  up  in  unstable,  low-paying  jobs,  and  little  was  known  about 
how  to  effectively  help  them  keep  employment  and  advance  in  their  jobs.  The  national  Employment  Retention 
and  Advancement  (ERA)  project  sought  to  fill  this  knowledge  gap,  by  examining  over  a dozen  innovative  and 
diverse  employment  retention  and  advancement  models  developed  by  states  and  localities  for  different  target 
groups,  to  determine  whether  effective  strategies  could  be  identified. 

Using  a random  assignment  research  design,  the  ERA  project  tested  the  effectiveness  of  programs  that 
attempted  to  promote  steady  work  and  career  advancement  for  current  and  former  welfare  recipients  and  other 
low-wage  workers,  most  of  whom  were  single  mothers.  The  programs  — generally  supported  by  existing 
public  funding,  not  special  demonstration  grants  — reflected  state  and  local  choices  regarding  target  popula- 
tions, goals,  ways  of  providing  services,  and  staffing.  The  ERA  project  is  being  conducted  by  MDRC,  under 
contract  to  the  Administration  for  Children  and  Families  (ACF)  in  the  U.S.  Department  of  Health  and  Human 
Services,  with  additional  funding  from  the  U.S.  Department  of  Labor.  This  report  presents  the  final  effective- 
ness findings,  or  impacts,  for  12  of  the  16  ERA  programs,  and  it  also  summarizes  how  the  12  programs  were 
implemented  and  individuals’  levels  of  participation  in  program  services. 


Key  Findings 

• Out  of  the  twelve  programs  included  in  the  report,  three  ERA  programs  produced  positive  econom- 
ic impacts;  nine  did  not.  All  three  programs  increased  employment  retention  and  advancement.  Increases 
in  employment  retention  and  earnings  were  largest  and  most  consistent  over  time  in  the  Texas  ERA  pro- 
gram in  Corpus  Christi  (one  of  three  sites  that  operated  this  program);  the  Chicago  ERA  program;  and  the 
Riverside  County,  California,  Post-Assistance  Self-Sufficiency  (PASS)  ERA  program.  These  programs 
increased  annual  earnings  by  between  7 percent  and  15  percent  relative  to  control  group  levels.  Each  of 
them  served  a different  target  group,  which  suggests  that  employment  retention  and  advancement  pro- 
grams can  work  for  a range  of  populations.  However,  three-fourths  of  the  ERA  programs  included  in  this 
report  did  not  produce  gains  in  targeted  outcomes  beyond  what  control  group  members  were  able  to  attain 
on  their  own  with  the  existing  services  and  supports  available  in  the  ERA  sites. 

• Increases  in  participation  beyond  control  group  levels  were  not  consistent  or  large,  which  may  have 
made  it  difficult  for  the  programs  to  achieve  impacts  on  employment  retention  and  advancement. 

Engaging  individuals  in  employment  and  retention  services  at  levels  above  what  they  would  have  done  in 
the  absence  of  the  programs  was  a consistent  challenge.  In  addition,  staff  had  to  spend  a lot  of  time  and 
resources  on  placing  unemployed  individuals  back  into  jobs,  which  made  it  difficult  for  them  to  focus  on 
helping  those  who  were  already  working  to  keep  their  jobs  or  move  up. 

Before  the  ERA  project  began,  there  was  not  much  evidence  about  the  types  of  programs  that  could  improve 
employment  retention  and  advancement  outcomes  for  current  or  former  welfare  recipients.  The  ERA  evalua- 
tion provides  valuable  insights  about  the  nature  of  retention  and  advancement  problems  and  it  underscores  a 
number  of  key  implementation  challenges  that  a program  would  have  to  address.  In  addition,  it  reveals 
shortcomings  in  a range  of  common  approaches  now  in  use,  while  identifying  three  distinct  approaches  that 
seem  promising  and  worthy  of  further  exploration. 
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About  the  Employment  Retention  and 
Advancement  Project 


The  federal  welfare  overhaul  of  1996  ushered  in  myriad  policy  changes  aimed  at  getting 
low-income  parents  off  public  assistance  and  into  employment.  These  changes  — especially 
cash  welfare’s  transformation  from  an  entitlement  into  a time-limited  benefit  contingent  on 
work  participation,  in  the  form  of  Temporary  Assistance  for  Needy  Families  (TANF)  — have 
intensified  the  need  to  help  low-income  families  become  economically  self-sufficient  and 
remain  so.  Although  a fair  amount  is  known  about  how  to  help  welfare  recipients  prepare  for 
and  find  jobs,  the  Employment  Retention  and  Advancement  (ERA)  project  is  the  most  compre- 
hensive effort  thus  far  to  ascertain  which  approaches  help  welfare  recipients  and  other  low- 
income  people  stay  steadily  employed  and  advance  in  their  jobs.  The  study  was  conceived  and 
funded  by  the  Administration  for  Children  and  Families  in  the  U.S.  Department  of  Health  and 
Human  Services;  supplemental  support  has  been  provided  by  the  U.S.  Department  of  Labor. 
The  evaluation  is  being  conducted  by  MDRC. 

Launched  in  1999,  the  ERA  project  encompasses  more  than  a dozen  models  and  uses  a 
rigorous  research  design  to  analyze  the  programs’  implementation  and  impacts  on  research 
sample  members. 1 In  total,  over  45,000  individuals  were  randomly  assigned  to  research  groups 
— in  each  site,  to  either  a program  group,  which  received  ERA  services,  or  a control  group, 
which  did  not  — starting  in  2000  in  the  earliest-starting  test  and  ending  in  2004  in  the  latest- 
starting  test.  The  random  assignment  process  ensured  that  when  individuals  entered  the  study, 
there  were  no  systematic  differences  in  sample  members’  characteristics,  measured  or  unmea- 
sured, between  the  program  and  control  groups  in  each  site.  Thus,  any  differences  between  them 
that  emerge  after  random  assigmnent  (for  example,  in  employment  stability  or  average  earn- 
ings) can  be  attributed  to  a site’s  ERA  program  — in  contrast  to  the  services  and  supports 
already  available  in  the  site.  These  differences  are  known  as  “impacts.” 

The  aims,  target  populations,  and  services  of  the  programs  studied  in  ERA  varied: 

• Advancement  programs  focused  on  helping  low-income  workers  (in  most 
cases,  workers  currently  or  recently  receiving  welfare)  move  into  better  jobs 
by  offering  such  services  as  career  counseling  and  education  and  training. 


'Sixteen  different  ERA  models  were  implemented  and  studied  in  eight  states:  California,  Illinois,  Minne- 
sota, New  York,  Ohio,  Oregon,  South  Carolina,  and  Texas. 
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• Placement  and  retention  programs  sought  to  help  participants  find  and 
hold  jobs  and,  in  some  cases,  were  aimed  at  “harder-to-employ”  people,  such 
as  welfare  recipients  who  had  disabilities  or  substance  abuse  problems. 

• Mixed-goals  programs  focused  on  job  placement,  retention,  and  advance- 
ment — in  that  order  — and  were  targeted  primarily  to  welfare  recipients 
who  were  searching  for  jobs. 

Prior  ERA  project  reports  describe  the  implementation  and  impacts  of  each  ERA  pro- 
gram, drawing  on  administrative  and  fiscal  records,  surveys  of  study  sample  members,  and  field 
visits  to  the  participating  sites,  as  well  as  using  the  strong  random  assigmnent  designs  (also 
known  as  “experimental”  designs)  embedded  in  each  ERA  model  test.  These  reports  address 
such  questions  as:  What  services  were  provided  by  the  program?  How  were  the  services 
delivered?  Who  received  them?  How  were  implementation  and  operational  problems  ad- 
dressed? To  what  extent  did  the  program  improve  employment  rates,  job  retention,  advance- 
ment, and  other  key  outcomes  in  the  short  run? 

This  report  presents  the  final  effectiveness  findings,  or  impacts,  for  12  of  the  16  ERA 
programs  and  also  summarizes  the  12  programs’  implementation  and  levels  of  participation  in 
program  services.  In  addition,  as  the  “report  of  record”  on  the  economic  impacts  of  most  ERA 
programs,  this  report  provides  the  foundation  for  future  additional  analyses.  The  report  thus 
provides  valuable  information  about  which  approaches  are  most  effective  as  well  as  the  building 
blocks  for  future  analyses  that  will  seek  to  ascertain  how  best  to  improve  employment  retention 
and  advancement  for  low-income  individuals. 
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Executive  Summary 


This  report  summarizes  the  final  impact  results  for  the  national  Employment  Retention 
and  Advancement  (ERA)  project.  This  project  tested,  using  a random  assignment  design,  the 
effectiveness  of  numerous  programs  intended  to  promote  steady  work  and  career  advancement. 
All  the  programs  targeted  current  and  former  welfare  recipients  and  other  low-wage  workers, 
most  of  whom  were  single  mothers.  Given  that  earlier  retention  and  advancement  initiatives 
studied  for  these  groups  were  largely  not  effective,  ERA  sought  to  examine  a variety  of  pro- 
grams that  states  and  localities  had  developed  for  different  populations,  to  determine  whether 
effective  strategies  could  be  identified.  In  short,  nine  of  the  twelve  programs  examined  in  this 
report  do  not  appear  to  be  effective,  but  three  programs  increased  employment  levels,  employ- 
ment stability,  and/or  earnings,  relative  to  control  group  levels,  after  three  to  four  years  of 
follow-up. 


The  ERA  Project:  Origin  and  Context 

Research  completed  since  the  1980s  has  resulted  in  much  being  learned  about  how  to 
help  welfare  recipients  and  other  low-income  individuals  prepare  for  and  find  jobs.  While  job 
preparation  and  placement  programs  have  improved  employment  and  earnings  among  partici- 
pants, they  primarily  moved  individuals  into  unstable,  low-paying  jobs.  Moreover,  research 
prior  to  ERA  that  examined  programs  explicitly  designed  to  increase  employment  retention 
among  newly  employed  welfare  recipients  found  that  the  programs  had  few  effects  on  retention 
or  advancement. 

The  ERA  project  was  intended  to  build  and  improve  on  past  efforts  in  the  employment 
retention  and  advancement  area.  It  was  conceived  and  funded  by  the  Administration  for 
Children  and  Families  (ACF)  in  the  U.S.  Department  of  Health  and  Human  Services  (HHS). 
The  project  was  also  supported  by  the  U.S.  Department  of  Labor  (DOL).  MDRC  — a nonprofit, 
nonpartisan  research  organization  — is  conducting  ERA  under  contract  to  ACF. 

From  2000  to  2003,  a total  of  16  innovative  ERA  programs  were  implemented  in  eight 
states  as  part  of  ERA.  This  report  presents  effectiveness  estimates  for  12  of  the  16  ERA  pro- 
grams and  for  the  main  group  of  individuals  targeted  for  services:  low-income  single  parents. 1 
The  results  cover  a three-  to  four-year  follow-up  period  and  examine  the  programs’  effects  on 
employment  retention,  earnings,  advancement  (defined  here  primarily  as  earnings  increases  not 
fully  explained  by  employment  increases),  and  the  receipt  of  welfare  and  food  stamp  benefits. 

'Findings  for  two-parent  families  served  by  these  programs  and  for  programs  that  targeted  “harder-to- 
employ”  individuals  are  not  included  here  but  will  be  presented  in  other  reports. 
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This  impact  report  is  the  document  of  record  for  the  ERA  programs’  impacts.  Its  pur- 
pose is  to  describe  what  was  tested  and  what  was  found.  A series  of  special  topic  papers  and 
practitioner  briefs  will  further  examine  the  results  contained  in  this  document,  to  try  to  provide 
more  clues  as  to  why  some  programs  may  have  had  positive  effects  and  others  did  not  and  to 
offer  lessons  for  policy  and  practice. 


The  ERA  Programs 

The  ERA  programs  embodied  states’  and  localities’  choices  of  program  goals,  target 
populations,  and  program  components  and  were  largely  paid  for  through  existing  funding 
streams.  The  programs  were  thus  “real-world”  ones  initiated  by  practitioners  and  not  ones  set  up 
and  funded  solely  for  research  purposes.  The  diversity  of  the  programs  presents  an  opportunity 
to  explore  the  effectiveness  of  a variety  of  strategies  implemented  for  different  populations,  in 
order  to  identify  what  might  work.  The  programs’  target  groups  and  general  strategies  are 
briefly  described  below.  Table  ES.l  provides  a summary  of  the  points  of  service  provision, 
selected  ways  of  delivering  services,  and  selected  types  of  services  for  each  of  the  ERA  pro- 
grams included  in  the  report. 

Target  Groups 

Almost  all  the  ERA  programs  targeted  current  or  former  recipients  of  Temporary  Assis- 
tance for  Needy  Families  (TANF),  the  cash  welfare  program  that  mainly  serves  single  mothers 
and  their  children.  The  programs  differed,  however,  in  terms  of  when  services  were  first 
provided  and  to  whom.  In  this  report,  findings  are  presented  in  three  groupings  of  programs, 
reflecting  the  programs’  target  populations:  (1)  programs  that  served  unemployed  TANF 
recipients,  (2)  programs  that  served  employed  TANF  recipients,  and  (3)  programs  that  served 
individuals  who  were  employed  and  not  receiving  TANF.  Presenting  the  results  in  these  three 
target  groupings  allows  practitioners  who  are  planning  to  implement  a program  for  a specific 
target  population  to  easily  see  the  range  of  ERA  program  models  that  were  implemented  for  that 
population  and  the  programs’  results.  In  addition,  it  facilitates  an  assessment  of  how  well  similar 
or  different  strategies  worked  in  achieving  similar  goals  for  roughly  similar  populations,  and  it 
suggests  whether  ERA  programs  work  better  when  initiated  at  one  or  another  point  in  individu- 
als’ employment  careers  — for  example,  when  people  are  looking  for  work  or  when  they  are 
already  in  work. 
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The  Employment  Retention  and  Advancement  Project 
Table  ES.l 

ERA  Models:  Overview  of  Target  Populations, 

Selected  Ways  of  Delivering  Services,  and  Selected  Types  of  Services 


Target  Populations 


Not  employed  and 
receiving  TANF 

Employed  and 
receiving  TANF 

Employed  and 
not  receiving  TANF 
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Wavs  of  delivering  services 

Nongovernmental  organization  as  a provider  of  services  V ■</  V •/ 

V 

V V V V 

Flexible  staff  hours  and/or  locations  V V V 

V V V 

V V V 

Employer  linkages  ,/  y/ 

Tvnes  of  services 

V 

v 

Financial  work  incentives  V V V 

V 

V 

Supported  advancement  through  job  change  y y 

V 

V 

Education  and  training  referrals  and/or  incentives 

V V V V 

V V V V 

Staff  counseling  on  job-related  issues  V V V V •/ 

V V V V 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E in  the  complete  report. 


NOTES:  For  discussion  of  the  full  set  of  features,  by  model,  see  Chapters  3,  4,  and  5.  For  the  definitions  of  these  features,  see  Chapter  2. 

Check  marks  indicate  that  the  feature  is  present  in  the  ERA  model. 

The  models  and  their  implementation  often  evolved  over  the  study  period.  This  table  presents  the  features  experienced  by  the  majority  of  the  study 
participants  for  the  greatest  extent  of  time. 

The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  all  operated  the  Texas  ERA  model. 


General  Strategies 

The  different  strategies  adopted  by  states  and  localities  to  promote  employment  reten- 
tion and  advancement  were  developed  in  accordance  with  program  operators’  views  of  the 
primary  impediments  to  retention  and  advancement  faced  by  individuals  in  their  chosen  target 
populations.  None  of  the  strategies,  however,  attempted  to  address  labor  market,  or  demand- 
side,  issues.  Rather,  they  all  sought  to  address  supply-side,  or  “worker-based,”  obstacles  to 
economic  success.  Within  the  broad  category  of  worker-based  strategies,  the  ERA  programs 
delivered  services  in  new  ways  and/or  provided  new  types  of  services  or  new  combinations  of 
services  compared  with  what  was  already  available  at  the  time. 

Delivering  Services  in  New  Ways 

A number  of  programs  provided  services  through  partnerships  between  welfare 
agency  staff  and  staff  from  other  organizations,  such  as  community  colleges,  Workforce 
Investment  Act  (WIA)  One-Stop  contractors,  nonprofit  employment  service  providers,  and/or 
community-based  organizations,  depending  on  the  program.  The  rationales  for  using  these 
service  delivery  arrangements  were  severalfold:  partnerships  increased  the  chances  of  leverag- 
ing resources;  these  arrangements  could  bring  together  different  types  of  services  and  expertise 
in  one  location,  and  thus  better  serve  program  participants;  and  it  was  thought  that  nongovern- 
mental organizations  might  be  more  familiar  than  governmental  agencies  with  the  jobs  and 
services  available  in  a community  and,  furthermore,  that  individuals  might  be  more  likely  to 
engage  with  staff  from  nonwelfare  organizations  after  leaving  TANF.  Many  ERA  programs 
also  implemented  new  staff  practices,  giving  staff  more  flexibility  in  when  and  where  to  meet 
with  clients,  to  make  it  easier  for  working  program  participants  to  meet  with  staff. 

Finally,  a few  ERA  programs  sought  to  develop  linkages  with  employers.  These  lin- 
kages took  the  form  of  engaging  a for-profit  company  that  had  strong  relationships  with  local 
employers,  to  facilitate  the  identification  of  openings  in  higher-paying  jobs;  meeting  with  clients 
at  their  employer  and  sometimes  with  clients’  employers,  to  discuss  opportunities  for  advance- 
ment; or  situating  ERA  services  at  individuals’  places  of  employment  to  make  it  easy  and 
convenient  for  people  to  participate  in  retention-related  services. 

Delivering  New  Types  of  Services  or  New  Combinations  of  Services 

The  ERA  programs  provided  some  common  services,  most  of  which  have  been  offered 
in  previous  and  current  programs.  All  the  programs  used  one-on-one  staff-client  interactions  as 
the  means  by  which  program  services  were  delivered,  although  they  differed  regarding  how  this 
was  done,  that  is,  what  was  discussed  during  staff-client  interactions,  the  intensity  of  the 
interactions,  and  staff  configurations  and  roles.  In  addition,  job  search  assistance  of  some  type 
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was  provided  in  almost  all  the  programs,  to  initially  get  individuals  into  jobs,  to  get  them  new 
jobs  if  they  left  or  lost  a job,  and/or  to  find  people  better  jobs  while  they  were  working.  In-depth 
assessments  of  individuals’  needs,  interests,  and  employment  barriers  were  also  done  in  many 
sites,  and  referrals  were  made  to  mental  health  or  substance  abuse  treatment  or  counseling. 

All  the  programs,  however,  experimented  with  providing  new  types  of  services  or  com- 
binations of  services.  Several  programs  provided  financial  incentives  or  rewards  — of  a 
substantial  amount  in  only  one  program  — connected  to  employment,  to  encourage  employ- 
ment entry,  promote  employment  retention,  and  raise  individuals’  implicit  wage  levels.  Other 
programs  provided  encouragement  and  support  to  change  jobs,  from  lower-paying  jobs  to 
higher-paying  ones  or  to  ones  with  better  opportunities  for  advancement.  A number  of  programs 
provided  encouragement  for  employed  individuals  to  participate  in  education  and  train- 
ing, under  the  theory  that  working  single  parents  may  need  support  to  engage  and  persist  in 
education  or  training,  given  the  difficultly  of  balancing  parenting,  work,  and  schooling.  Finally, 
many  of  the  programs  provided  individuals  with  counseling  on  job-related  issues,  under  the 
expectation  that  this  could  both  prevent  unnecessary  job  loss  and  educate  workers  on  how  to 
advance  at  their  current  job. 


The  Evaluation’s  Design 

Each  of  the  ERA  programs  discussed  in  this  report  was  studied  using  a random  assign- 
ment evaluation  design,  a methodology  that  allows  practitioners  and  policymakers  to  have  a 
high  degree  of  confidence  in  the  results.  In  each  site,  individuals  who  met  the  ERA  eligibility 
criteria  (which  varied  by  site)  were  assigned,  at  random,  to  a program  group  or  to  a control 
group.  Members  of  the  program  group  were  recruited  for  (and,  in  some  sites,  were  required  to 
participate  in)  the  services  offered  by  the  ERA  program,  while  those  in  the  control  group  were 
not  eligible  for  ERA  services  but  were  eligible  for  other  services  and  supports.  The  control 
group  services  and  supports  were  always  those  generally  available  in  the  site’s  community  but 
could  also  include  the  site’s  standard  welfare-to-work  program  or,  in  some  cases,  minimal 
efforts  that  the  site  already  had  in  place  to  provide  assistance  to  individuals  who  found  jobs.  As 
a result,  none  of  the  control  groups  can  be  strictly  considered  to  be  a “no  service”  group.  Each 
site’s  control  group  thus  represents  the  benchmark  against  which  that  site’s  ERA  approach  is 
assessed.2 


2There  was  significant  variation  across  sites  in  terms  of  the  types  of  services  for  which  control  group 
members  were  eligible,  the  extent  of  services,  and  how  well  known  and  readily  accessible  the  services  were. 
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In  total,  over  45,000  individuals  were  randomly  assigned  to  research  groups  as  part  of 
ERA,  starting  in  2000  in  the  earliest-starting  test  and  ending  in  2004  in  the  latest-starting  test.3 
In  each  site,  MDRC  tracked  individuals  in  both  research  groups  for  three  to  four  years  following 
their  random  assignment,  using  administrative  records  — such  as  unemployment  insurance 
(UI),  welfare,  and  food  stamp  records  — and  sample  member  surveys.  The  random  assignment 
process  ensured  that  when  individuals  entered  the  study,  there  were  no  systematic  differences  in 
sample  members’  characteristics,  measured  or  umneasured,  between  the  research  groups.  Thus, 
any  differences  between  them  that  emerge  after  random  assignment  (for  example,  in  employ- 
ment rates  or  average  earnings)  can  be  attributed  to  the  site’s  ERA  program  — in  contrast  to 
services  and  supports  already  available  in  the  site.  These  differences  are  referred  to  here  as 
“impacts.” 


Key  Findings 

The  implementation,  participation,  and  economic  impact  analyses  in  this  report  yielded 
the  following  key  findings. 

Implementation  Findings 

• Engaging  individuals  in  employment  and  retention  services  was  a con- 
sistent challenge. 

Staff  in  the  ERA  programs  expended  considerable  energy  trying  to  engage  individuals 
in  program  activities.  Many  ERA  programs,  particularly  those  targeting  individuals  outside  the 
TANF  system,  included  intensive  marketing  and  outreach  strategies.  The  ERA  programs  also 
used  a variety  of  strategies  to  maintain  staff-client  relationships  and  encourage  ongoing  partici- 
pation in  program  activities,  including,  in  some  programs,  offering  financial  incentives  to 
encourage  contact  and  working  with  individuals  at  their  workplaces.  However,  while  most 
program  group  members  reported  having  at  least  one  contact  with  ERA  program  staff,  main- 
taining contact  over  the  course  of  the  first  year  following  random  assignment  was  less  common. 
Only  five  programs  increased  the  percentage  of  program  group  members,  compared  with 
control  group  members,  who  were  having  contact  with  staff  from  an  employment  program  as  of 
the  end  of  the  first  year  of  follow-up.  Moreover,  even  in  these  five  programs,  increases  in 
ongoing  engagement  were  modest. 


’Outcomes  for  approximately  27,000  sample  members  are  analyzed  in  this  report.  Other  documents  will 
present  results  for  the  remaining  sample  members. 
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• Rates  of  job  loss  were  very  high,  and  job  loss  occurred  quickly  in  all  the 
ERA  programs,  requiring  staff  to  spend  a significant  amount  of  their 
time  providing  reemployment  services. 

In  a number  of  the  ERA  programs  that  attempted  to  provide  services  to  individuals 
when  they  were  employed  (referred  to  as  “postemployment  programs”  in  this  report),  high 
levels  and  fast  rates  of  job  loss  had  several  implications.  One  implication  was  that  staff  had  less 
time  available  to  work  with  individuals  on  advancement  services.  Another  implication  was  that 
the  volume  of  demands  for  reemployment  services  disrupted  programs’  staffing  plans.  Staff  in 
postemployment  programs  were  prepared  for  and  anticipated  working  with  employed  clients  to 
aid  their  placement  into  better  jobs  or  ones  positioned  for  advancement.  Staff  were  also  pre- 
pared to  provide  employment  retention  services  to  attempt  to  prevent  job  loss.  In  practice,  many 
individuals  had  lost  their  jobs  by  the  time  program  staff  first  made  contact  with  them,  forcing 
staff  to  focus  on  immediate  placement  needs  instead. 

• The  staffing  of  the  ERA  programs  was  critical,  as  most  services  hinged 
on  staff  actions  and  many  welfare  agency  staff  lacked  skills  and  expe- 
rience relevant  to  retention  and  advancement  aims. 

In  most  of  the  ERA  programs,  staff  were  responsible  for  providing  a range  of  services, 
rather  than  specializing  in  one  or  a few.  Many  of  these  services  — such  as  recruitment,  career 
counseling,  labor  market  assessment,  and  job  development  — were  not  typically  provided  in 
standard  welfare-to-work  programs.  Thus,  the  ERA  programs  often  demanded  skill  sets  and 
facilities  in  arenas  or  with  populations  that  were  new  to  staff.  Despite  efforts  to  hire  and  train 
qualified  staff,  providing  the  various  retention  and  advancement  services  was  a struggle  for 
many  ERA  program  staff. 

Participation  Findings 

• Increases  in  participation  beyond  control  group  levels  were  not  consis- 
tent or  large,  which  may  have  made  it  difficult  for  the  programs  to 
achieve  impacts  on  employment  retention  and  advancement. 

Overall,  most  of  the  ERA  programs  produced  relatively  small  increases  in  participation 
in  program  activities  and  in  the  receipt  of  various  types  of  retention  and  advancement  assis- 
tance, but  the  reasons  for  this  differed  by  the  type  of  activity  or  “help”  provided  in  the  pro- 
grams. Many  of  the  control  groups  in  this  study  were  eligible  for  welfare-to-work  programs  that 
often  provided  such  services  as  job  search  and  education  and  training  referrals.  Therefore,  the 
control  group  levels  of  participation  in  these  services  was  often  high  (with  participation  levels  in 
education  and  training  being  unexpectedly  high  in  the  case  of  programs  serving  employed 
individuals),  and  few  ERA  programs  significantly  raised  participation  above  these  levels. 
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In  addition  to  encouraging  participation  in  job  search  and  education  and  training,  most 
ERA  programs  offered  other  types  of  retention  and  advancement  assistance,  such  as  help 
resolving  problems  on  the  job  or  help  finding  a better  job  while  working.  While  a 12-month 
client  survey  showed  that  few  control  group  members  received  these  types  of  assistance,  few 
ERA  programs  increased  the  levels  of  receipt  of  these  services  by  a substantial  amount  above 
the  control  group  levels. 

While  the  increases  in  participation  in  ERA  services  in  a number  of  program  tests  were 
infrequent  and  small  for  two  different  reasons  — high  levels  of  control  group  participation  in 
some  activities  and  low  levels  of  program  group  participation  in  others  — the  end  result  was  the 
same:  little  impact  on  participation  in  services  in  these  tests  made  it  unlikely  that  these  particular 
programs  would  improve  economic  outcomes  (though  there  is  no  guarantee  that  these  programs 
would  have  improved  economic  outcomes  even  if  they  had  been  able  to  boost  participation 
more  substantially).  A few  programs,  however,  did  produce  some  sizable  participation  increases 
and  also  increased  employment  retention  and  advancement,  as  discussed  in  more  detail  below. 

Effectiveness  Findings 

Several  types  of  economic  effects  are  examined  in  this  report.  Effects  on  employment 
retention  (defined  here  as  impacts  on  the  extent  of  individuals’  labor  force  participation)  are 
primarily  based  on  two  key  outcomes,  both  using  UI  data:  the  average  quarterly  employment 
rate  and  the  extent  to  which  people  worked  in  four  consecutive  quarters.  Any  impacts  on  the 
average  quarterly  employment  rate  would  indicate  that  program  group  members,  compared  with 
control  group  members,  worked  in  more  quarters.  Any  impacts  on  working  four  consecutive 
quarters  would  indicate  that  program  group  members,  compared  with  control  group  members, 
were  more  likely  to  remain  consistently  employed,  though  not  necessarily  in  the  same  job. 

Effects  on  advancement  are  defined  here  primarily  in  terms  of  increases  in  earnings  that 
cannot  be  fully  explained  by  increases  in  quarters  of  employment.  When  a program  increases 
total  earnings,  it  can  reflect  either  or  both  the  program’s  effect  on  employment  retention  and  its 
effect  on  advancement.  Any  increases  in  total  earnings  would  indicate  that  program  group 
members,  compared  with  control  group  members,  had  more  quarters  of  employment,  worked 
more  hours  or  weeks,  or  worked  at  higher  wages.  Because  most  advancement  measures  are 
created  using  UI  data,  it  is  impossible  to  separate  out  how  much  of  the  increase  in  earnings  was 
due  to  more  hours  or  weeks  or  working  at  higher  wages,  but  it  is  possible  to  estimate  how  much 
was  due  to  an  increase  in  quarters  of  employment.  Therefore,  if  the  increases  in  total  earnings 
are  not  fully  explained  by  the  effects  on  the  number  of  quarters  employed,  it  is  likely  that 
“advancement”  (defined  as  individuals’  working  more  hours  or  weeks  or  working  at  higher 
wages)  occurred.  In  this  report,  advancement  is  secondarily  defined  as  improvements  in  fringe 
benefits  received,  working  conditions,  or  opportunities  for  promotions. 
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Table  ES.2  indicates  which  ERA  programs  produced  impacts  on  three  of  the  above  out- 
comes: average  quarterly  employment,  working  four  consecutive  quarters,  and  average  annual 
earnings.  (Survey-based  impacts  are  not  shown.)  Only  statistically  significant  impacts  are 
shown  in  the  table  and  are  discussed  in  this  summary  and  the  report,  except  where  otherwise 
indicated.4 

• Out  of  the  twelve  ERA  programs  included  in  the  report,  three  programs 
— one  in  each  target  grouping  — produced  positive  economic  impacts: 
the  Texas  (Corpus  Christi  and  Fort  Worth  sites),  Chicago,  and  River- 
side PASS  ERA  programs. 

The  Texas  ERA  program  offered  a monthly  stipend  of  $200  for  former  TANF  reci- 
pients working  at  least  30  hours  per  week,  in  addition  to  other  pre-  and  postemployment 
services.  As  implemented  in  Corpus  Christi,  the  program  had  consistent  effects  on  employment 
retention  and  earnings.  There  is  also  evidence  that  the  program  may  have  led  to  advancement 
gains  — meaning  gains  in  hours  worked,  weeks  worked,  or  wages  — compared  with  the  level 
of  advancement  seen  in  the  control  group.  (Increases  in  earnings  could  not  be  fully  explained  by 
increases  in  the  number  of  quarters  employed.)  In  Corpus  Christi,  the  program  increased 
average  annual  earnings  by  $640  over  the  four-year  follow-up  period,  or  by  almost  15  percent 
relative  to  control  group  earnings.  In  addition,  the  program  generated  its  largest  effects  on 
earnings  in  the  fourth  year  of  follow-up,  when  it  increased  earnings  by  $900,  or  by  18  percent 
relative  to  the  control  group  level.  This  suggests  that  the  impacts  may  continue  in  the  longer 
term.  The  Texas  ERA  program  in  Fort  Worth  also  produced  increases  in  these  measures,  but  the 
effects  were  concentrated  in  the  second  and  third  years  of  follow-up.  In  Fort  Worth,  the  pro- 
gram increased  earnings  in  the  third  year  of  follow-up  by  $900,  or  by  17  percent  relative  to  the 
control  group  level.  The  implementation  of  the  Texas  program  in  Fort  Worth  started  out  weak 
and  improved  over  time,  which  may  have  diluted  the  strength  of  the  impacts  because  many 
program  group  members  went  through  the  program  before  implementation  improved. 

The  program  in  Chicago  was  a mandatory,  work-focused  advancement  program,  offer- 
ing targeted  job  search  assistance  and  help  to  identify  and  access  career  ladders,  provided  by 
staff  in  a private,  for-profit  firm.  This  program  produced  increases  in  employment  retention  and 
earnings.  In  addition,  there  is  evidence  that  the  program  may  have  led  to  advancement  gains, 
compared  with  the  level  of  advancement  seen  in  the  control  group.  But  while  the  Chicago  ERA 
program  raised  average  annual  earnings  by  almost  $500,  or  by  7 percent  relative  to  the  control 
group  level,  these  effects  weakened  over  time.  Impacts  on  employment  were  no  longer  statisti- 


4An  impact  is  considered  to  be  statistically  significant  if  there  is  less  than  a 10  percent  probability  that  the 
estimated  difference  would  have  occurred  by  chance  in  the  absence  of  any  effect  of  the  program.  The  tables 
and  figures  in  this  report  note  statistical  significance  at  the  10  percent,  5 percent,  and  1 percent  levels. 
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Table  ES.2 


ERA  Models:  Summary  of  Economic  Impact  Results 


Model 

Average  Quarterly 
Employment 

Employed  4 Consecutive 
Quarters 

Average  Annual  Earnings 

Y1 

Cumulative 
Y2  Y3  Y4  follow-up 

Y 1 Y2 

Cumulative 
Y3  Y4  follow-up 

Y1  Y2 

Y3 

Y4 

Cumulative 

follow-up 

Not  employed  and  receiving  TANF 

Texas 

Corpus  Christi 

• 

• • • 

• 

• • 

• • 

• 

• 

• 

Fort  Worth 

• 

• 

• 

Houston 

Los  Angeles  EJC 

— 

• 

— 

— 

Salem 

— 

— 

— 

Employed  and  receiving  TANF 

Chicago 

• 

• • 

• 

• 

• 

Los  Angeles  RFS 

— 

— 

— 

Riverside  Phase  2 

Riverside  Training  Focused"1 

Riverside  Work  Plus 

Employed  and  not  receiving  TANF 

Cleveland 

— 

— 

— 

Eugene3 

— 

— 

Medford 

— 

— 

— 

Riverside  PASS 

• 

• • • • 

• 

• • 

• • 

• 

• 

• 

South  Carolina 

(continued) 


Table  ES.2  (continued) 

SOURCES:  MDRC  calculations  from  the  state  administrative  records. 

NOTES:  A dash  indicates  “not  available”  or  that  the  sample  size  is  too  small  to  yield  meaningful  results. 

Bullets  indicate  positive  statistically  significant  differences  between  outcomes  for  the  program  and  control 
groups  up  to  the  10  percent  level. 

aThe  negative  impacts  for  the  Eugene  and  Riverside  Training  Focused  programs  are  not  shown  on  this 
table.  For  negative  impacts,  see  the  site-specific  summary  tables  in  the  complete  report. 

cally  significant  in  Year  4 as  a whole,  but  positive  earnings  impacts  persisted  in  some  quarters, 
suggesting  that  the  Chicago  program  may  have  had  a long-term  effect  on  advancement.  It  also 
achieved  the  largest  reductions  in  welfare  receipt  among  all  the  ERA  programs.  Program  group 
levels  of  welfare  receipt  were  approximately  25  percent  lower  than  control  group  levels. 

Increases  in  employment  retention  and  earnings  were  large  and  consistent  in  the  River- 
side PASS  ERA  program,  which  was  a voluntary,  individualized,  retention  and  advancement 
program  provided  primarily  by  three  community-based  organizations  and  a community  college. 
There  is  also  evidence  that  the  program  may  have  led  to  increases  in  advancement,  compared 
with  what  was  seen  in  the  control  group.  This  program  increased  average  annual  earnings  by 
$870  over  the  four-year  follow-up  period,  an  increase  of  10  percent  relative  to  the  control  group 
level.  In  addition,  the  program  generated  its  largest  effects  on  earnings  ($970)  in  the  fourth  year 
of  follow-up,  suggesting  that  the  program  may  lead  to  even  longer-term  earnings  gains. 

• In  the  Chicago  and  Riverside  PASS  ERA  programs,  which  provided 
services  to  employed  individuals,  impacts  were  driven  by  entering 
another  job  after  random  assignment  — not  by  staying  stably  employed 
at  the  original  job. 

Of  the  three  programs  that  produced  positive  impacts,  two  of  them  initiated  services  after 
people  had  found  jobs  (postemployment  programs),  and  one  of  them  (referred  to  here  as  a 
“preemployment  program”)  initiated  services  before  people  found  jobs.  It  might  be  expected  that 
the  postemployment  programs  would  achieve  their  impacts  by  lengthening  the  amount  of  time 
that  people  stayed  in  the  job  they  held  when  they  entered  the  program.  However,  in  both  of  these 
postemployment  programs,  the  impacts  reflect  participants’  entering  new  jobs  at  greater  rates  than 
control  group  members,  either  directly  from  another  job  or  after  a period  of  unemployment. 

This  result  suggests  that  even  programs  designed  to  affect  employment  retention  and  ad- 
vancement among  those  already  working  should  be  very  deliberate  about  how  job  change  and  job 
loss  are  addressed.  Job  changes  and  reemployment  needs  may  present  opportunities  to  improve 
outcomes.  Job  loss,  for  example,  can  be  analyzed  to  understand  the  reasons  why  people  lost  their 
job  and  can  lead  to  developing  plans  to  avoid  job  loss  in  the  future.  Similarly,  job  changes  can  be 
used  as  a way  to  place  individuals  into  better  jobs.  In  addition,  this  result  suggests  that  a focus  on 
employment  retention,  rather  than  job  retention,  may  be  more  effective  or  more  realistic. 
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Conclusions 

The  reports’  findings  suggest  several  conclusions. 

• The  results  revealed  some  strategies  that  can  be  effective  in  promoting 
employment  retention  and  advancement  for  welfare  recipients  and  oth- 
er low-income  groups. 

Increases  in  employment  retention  and  earnings  were  largest  and  most  consistent  over 
time  in  the  Texas  ERA  program  in  Corpus  Christi,  the  Chicago  ERA  program,  and  the  River- 
side PASS  ERA  program.  The  Texas  program  in  Fort  Worth  improved  over  time  and  produced 
gains  in  Years  2 and  3.  Each  of  these  three  ERA  programs  served  a different  target  group, 
which  suggests  that  employment  retention  and  advancement  programs  can  work  for  a range  of 
populations. 

Table  ES.3  highlights  the  features  of  these  three  programs.  Caution  should  be  exercised 
when  trying  to  identify  promising  program  features,  however,  as  some  of  the  programs  lacking 
positive  impacts  also  contained  some  of  these  features  and  it  is  not  clear  which  features  “drove” 
the  impacts  in  any  given  site.  Two  findings  from  the  research  literature  can  help  place  these 
findings  in  context.  With  regard  to  the  Texas  findings,  several  studies  have  now  shown  that 
programs  that  provide  financial  incentives  to  supplement  earnings  in  combination  with  services 
can  promote  employment  retention  among  low-wage  workers.5  With  regard  to  the  Chicago 
program,  nonexperimental  work  has  suggested  that  low-wage  workers  often  advance  by 
changing  jobs  and  that  matching  individuals  with  jobs  in  particular  finns  that  pay  higher  wages 
can  be  an  effective  strategy  to  promote  advancement.6  The  fact  that  these  strategies  have  been 
found  to  be  effective  in  previous  research  as  well  as  in  this  study  lends  additional  evidence  that 
these  are  promising  strategies  to  promote  employment  retention  and  advancement. 

• Improving  employment  retention  and  advancement  remains  a challenge. 

Three-fourths  of  the  ERA  programs  included  in  this  report  did  not  produce  gains  in  the 
targeted  outcomes  beyond  what  control  group  members  were  able  to  attain  on  their  own  with 
the  existing  set  of  services  and  supports.  This  suggests  that,  despite  a range  of  programs  and 
approaches  and  significant  effort  by  staff  and  program  managers,  gains  in  employment  retention 
and  advancement  are  difficult  to  attain.  In  addition,  even  among  the  programs  that  produced 
improvements  in  retention  and  advancement  outcomes,  wages  remained  very  low,  and  many  of 


5Gennetian,  Miller,  and  Smith  (2005);  Huston  et  al.  (2003);  Michalopoulos  et  al.  (2002);  Riccio  et  al. 
(2008). 

6Andersson,  Holzer,  and  Lane  (2005).  Indeed,  a recent  nonexperimental  analysis  of  the  employment  pat- 
terns of  ERA  sample  members  confirmed  that  job  change  was  associated  with  higher  wages  for  median- 
earning sample  members  (Miller,  Deitch,  and  Hill,  2010). 
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The  Employment  Retention  and  Advancement  Project 
Table  ES.3 

Highlights  of  Programs  That  Increased  Employment  Retention  and  Advancement 


Corpus  Christi,  Texas 
Fort  Worth,  Texas 

Chicago 

Riverside  PASS 

Strategy 

Placement,  retention,  and 
advancement:  job  search 
assistance,  stipend  for 
employed  fonner  TANF 
recipients,  reemployment 
assistance,  and  work  site 
visits 

Advancement:  job  search 
assistance,  career  counseling, 
and  reemployment  assistance 

Retention  and  advancement: 
reemployment  assistance, 
career  counseling,  and 
referrals  to  education  and 
training 

Target  population 

Unemployed  TANF 
applicants  and  recipients 

TANF  recipients  who  had 
worked  at  least  30  hours  per 
week  for  at  least  6 
consecutive  months 

Employed  fonner  TANF 
recipients  who  recently  left 
TANF 

Service  providers 

Local  workforce 
development  boards  under 
contract  with  nonprofit 
organizations 

Experienced,  for-profit, 
employment  intermediary 

Primarily  community-based 
organizations  and  a 
community  college 

Control  services 

Relatively  strong  welfare-to- 
work  program 

Standard  welfare-to-work 
program 

Limited  postemployment 
services 

Participation 

highlights 

In  Corpus  Christi,  30  percent 
took  up  the  financial 
incentive;  20  percent  did  so 
in  the  other  Texas  sites. 
Increased  percentages 
receiving  help  with  retention 
and  advancement  in  Corpus 
Christi  and  Fort  Worth. 

Increased  percentages 
receiving  help  in  finding  a 
better  job  while  working  and 
in  getting  other  forms  of 
retention  and  advancement 
help. 

While  increases  in 
participation  were  not  large, 
participation  data  are  for  a 
cohort  that  had  few  positive 
economic  impacts.3 

Economic  impacts 

Increased  employment 
retention  and  advancement  in 
Corpus  Christi  and  Fort 
Worth  but  not  in  Houston 

Increased  employment 
retention  and  advancement 
and  reduced  welfare  receipt 

Increased  employment 
retention  and  advancement 

SOURCES:  ERA  12-Month  Survey,  administrative  records,  and  interim  reports. 


NOTES:  “Participation  impacts  in  this  report  are  based  on  client  survey  data.  As  detailed  in  the  text,  survey 
response  issues  were  present  in  the  Riverside  PASS  site.  Analysis  of  program  data  suggested  that  only  the 
Riverside  PASS  program  may  have  increased  the  use  of  a broader  range  of  services.  These  data  are  not  used 
in  this  report  because  they  only  capture  services  reported  to  the  welfare  department  by  ERA  service 
providers. 
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these  families  remained  in  poverty  at  the  end  of  the  follow-up  period.  It  is  also  important  to 
consider,  however,  that  ERA  was  a test  of  multiple  strategies  rather  than  a demonstration  of  a 
few  promising  programs.  As  noted  above,  before  the  ERA  project  began,  there  was  not  much 
evidence  of  programs  that  could  improve  employment  retention  and  advancement  outcomes  for 
current  or  fonner  welfare  recipients.  The  ERA  evaluation  has  revealed  three  distinct  approaches 
that  seem  promising  and  worthy  of  further  exploration. 

* * * 

While  this  report  presents  the  final  impact  estimates  of  many  of  the  ERA  programs,  it  is 
not  the  final  word  from  the  ERA  project.  Further  analysis  is  being  conducted  of  the  rich  ERA 
databases  to  fry  to  shed  light  on  which  program  features,  participant  characteristics,  institutional 
arrangements,  or  economic  contexts  were  most  associated  with  improvements  in  employment 
retention  and  advancement.  In  addition,  other  publications  will  consider  the  costs  and  benefits 
of  selected  ERA  programs,  the  employment  and  training  dynamics  of  the  ERA  sample,  and  the 
employment  patterns  of  ERA  sample  members  in  two-parent  families.  A series  of  practice 
briefs  will  also  probe  more  deeply  and  extract  lessons  for  those  who  operate  and  staff  retention 
and  advancement  programs.  Finally,  a synthesis  document  will  consider  the  policy,  practice, 
and  research  lessons  learned  from  the  whole  of  the  ERA  project. 
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Chapter  1 

Introduction 


Since  the  mid-1990s,  there  has  been  growing  interest  among  policymakers  in  identify- 
ing the  kinds  of  services,  supports,  and  incentives  that  can  help  low-income  working  parents 
retain  steady  employment  and  advance  in  the  labor  market.  This  has  reflected  several  develop- 
ments. First,  broad  economic  trends  have  reduced  the  availability  of  high-paying  jobs  for  people 
who  do  not  have  a college  education  and  have  increased  the  number  of  families  headed  by  low- 
income  workers.  Partially  because  of  this,  the  federal  government  and  many  states  have  in- 
creased spending  on  supports  for  low-income  working  families,  such  as  tax  credits,  child  care 
subsidies,  and  health  insurance.  Second,  the  welfare  refonns  of  the  1990s  encouraged  or 
required  millions  of  single  parents  with  low  skills  (primarily  women)  to  enter  the  labor  market 
and  imposed  time  limits  on  the  receipt  of  assistance.  As  a result  of  these  developments,  long- 
term welfare  receipt  has  become  much  less  common,  and  steady  employment  and  wage 
advances  have  grown  in  importance  as  means  of  achieving  economic  well-being.  At  the  same 
time,  the  current  economic  climate  in  the  United  States  has  made  achieving  these  objectives 
more  difficult  by  increasing  the  number  of  individuals  at  risk  of  losing  employment  or  ad- 
vancement opportunities. 

As  of  1 0 years  ago,  little  was  known  about  effective  strategies  to  help  low-income  par- 
ents keep  employment  and  advance  in  their  jobs.  While  research  had  yielded  substantial 
knowledge  about  how  to  help  welfare  recipients  and  other  low-income  individuals  prepare  for 
and  find  jobs,  many  participants  in  these  successful  job  preparation  and  placement  programs 
ended  up  in  unstable,  low-paying  jobs.  Moreover,  prior  research  — in  the  mid-1990s  — that 
examined  programs  explicitly  designed  to  increase  employment  retention  among  newly 
employed  welfare  recipients  found  that  the  programs  had  few  effects  on  retention  or  advance- 
ment. 1 


The  Employment  Retention  and  Advancement  (ERA)  project  was  designed  to  fill  the 
gap  in  knowledge  about  employment  retention  and  advancement  strategies  that  might  be 
effective.  ERA’S  goal  was  to  identify  and  rigorously  test  a diverse  set  of  innovative  models 
designed  to  promote  employment  stability  and  wage  or  earnings  progression  among  current  or 
former  welfare  recipients  or  other  low-income  groups.  As  part  of  ERA,  over  a dozen  different 
program  models  have  been  evaluated  over  the  past  10  years  using  random  assignment  research 
designs.  These  models  embodied  states’  and  localities’  choices  of  program  goals,  target  popula- 


For  results  from  the  Post-Employment  Services  Demonstration  (PESD),  see  Rangarajan  and  Novak 
(1999). 
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tions,  and  program  features,  and  the  programs  were  largely  paid  for  through  existing  funding 
streams.  The  programs  were  thus  “real-world”  interventions  initiated  by  practitioners  and  not 
ones  set  up  and  funded  solely  for  research  purposes.  The  diversity  of  the  models  presents  an 
opportunity  to  explore  the  effectiveness  of  a variety  of  strategies  implemented  for  different 
populations,  in  order  to  identify  what  might  work. 

This  report  summarizes  long-term  findings  on  the  effects  of  12  of  the  ERA  program 
models  for  their  primary  target  group:  low-income,  generally  female,  single  parents.  (Findings 
for  two-parent  families  served  in  these  programs  and  for  the  programs  that  targeted  “harder-to- 
employ”  individuals  are  not  included  here  but  will  be  presented  in  other  reports.)  Table  1.1 
provides  a summary  of  the  targeted  populations  and  services  that  were  offered  in  the  12  ERA 
programs  covered  in  this  report. 

The  report’s  effectiveness  findings  are  based  on  a comparison  of  each  site’s  ERA  pro- 
gram and,  usually,  the  site’s  preexisting  services  and  supports,  which  often  included  the  site’s 
standard  welfare-to-work  program.  The  ERA  programs’  effects  thus  represent  the  added  value 
of  the  ERA  models  beyond  the  services  and  supports  already  available.  The  report  examines 
effects  on  employment  retention  (defined  here  as  the  extent  of  an  individual’s  labor  force 
participation),  on  earnings,  and  on  advancement  (defined  here  primarily  as  increases  in  earnings 
that  are  not  fully  explained  by  increases  in  quarters  of  employment).  Effects  on  use  of  welfare 
and  other  income  supports  are  also  examined. 

For  each  of  the  12  programs,  the  report  presents  implementation  findings  and  final  eco- 
nomic effects  measured  over  three  or  four  years.  As  such,  it  is  the  foundation  for  a number  of 
analytical  papers,  syntheses,  and  practice  briefs  that  will  be  forthcoming  from  the  ERA  project. 
Thus,  while  the  12  programs’  final  effectiveness  findings  are  included  here,  future  documents 
will  provide  additional  guidance  to  policymakers  and  program  operators  as  they  seek  to  glean 
lessons  from  the  ERA  project. 

In  brief,  the  findings  reported  here  are  mixed.  Many  strategies  do  not  appear  to  be  effec- 
tive, but  some  do  appear  to  increase  employment  retention  and,  in  some  cases,  advancement. 
Future  ERA  documents  will  seek  to  offer  hypotheses  regarding  why  some  of  the  strategies  may 
have  had  positive  effects  and  others  did  not,  as  well  as  lessons  for  policy  and  practice. 

This  chapter  provides  an  introduction  to  ERA  by  describing  the  project’s  history,  the 
models  tested  in  the  project,  ERA’S  research  design,  the  settings  of  the  ERA  tests,  ERA  sample 
members’  characteristics,  and  the  typical  labor  market  behavior  of  the  groups  targeted  in  ERA. 
Finally,  the  chapter  states  the  research  questions  addressed  in  this  report,  presents  considerations 
in  interpreting  ERA’s  effectiveness  results,  and  outlines  the  report’s  organization. 
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The  Employment  Retention  and  Advancement  Project 

Table  1.1 

Brief  Description  of  ERA  Programs 


Program 

Target  Group 

Program  Description 

Texas 

Corpus  Christi 
Fort  Worth 
Houston 

Unemployed  TANF  applicants  and 
recipients 

Mandatory  preemployment  job  search  assistance,  followed  by  voluntary  postemployment 
assistance  (which  could  include  employer  site  visits  and  reemployment  assistance),  with  a monthly 
stipend  of  $200  for  fonner  TANF  recipients  working  at  least  30  hours  per  week;  services  provided 
by  staff  in  nonprofit  organizations  contracted  by  local  workforce  agencies 

Los  Angeles 
Enhanced  Job  Club 

Unemployed  TANF  recipients 

Mandatory  5-week  job  club  focused  on  career  development  activities  and  job  search  targeted  to 
individuals’  careers  of  interest;  operated  jointly  by  county  welfare  and  education  agency  staff 

Salem  (Oregon) 

Unemployed  TANF  applicants 

Mandatory  preemployment  job  search  assistance  and  voluntary  postemployment  services;  jointly 
provided  by  welfare  agency  and  community  college  staff 

Chicago 

TANF  recipients  who  had  worked 
at  least  30  hours  per  week  for  at 
least  6 consecutive  months 

Mandatory  work-focused  advancement  program  offering  targeted  job  search  assistance  and  help 
to  identify  and  access  career  ladders;  provided  by  staff  in  a private,  for-profit  firm 

Los  Angeles  Reach 
for  Success 

TANF  recipients  who  had  worked 
at  least  32  hours  per  week  for  at 
least  30  days 

Voluntary,  intensely  marketed,  individualized  retention  and  advancement  program;  administered 
by  welfare  agency  staff 

Riverside  Training 
Focused 

TANF  recipients  who  had  worked 
at  least  20  hours  per  week  for  at 
least  30  days 

Education/training-focused  advancement  program  with  flexibility  to  reduce  or  eliminate  required 
work  hours  if  participating  in  education  or  training;  operated  by  workforce  agency 

Riverside  Work 

Plus 

TANF  recipients  who  had  worked 
at  least  20  hours  per  week  for  at 
least  30  days 

Education/training-focused  advancement  program  with  less  flexibility  (compared  to  the  Riverside 
Training  Focused  program)  to  reduce  or  eliminate  required  work  hours  if  participating  in 
education  or  training;  operated  by  county  welfare  agency 

(continued) 


Table  1.1  (continued) 


Program 

Target  Group 

Program  Description 

Cleveland 

Low-wage  workers  at  specific 
employers  who  earned  less  than 
$13  per  hour  and  who  had  been  in 
their  current  job  for  less  than  6 
months 

Voluntary,  employer-based  employment  retention  program,  including  ongoing  staff-client 
relationships,  weekly  peer  support  groups,  and  supervisory  trainings;  provided  by  a community- 
based  organization 

Eugene  (Oregon) 

Employed  individuals  who  had  left 
TANF  within  the  year  and  who 
were  working  more  than  20  hours 
per  week 

Voluntary,  individualized  program,  including  career  counseling  and  service  referals;  provided 
through  a welfare  agency  partnership  with  a community  college  and  WIA  contractor. 

Medford  (Oregon) 

Employed  individuals  who  were 
former  recipients  of  TANF  or 
current  recipients  of  the  Oregon 
Food  Stamp  Employment  and 
Training  program  and/or  the 
Employment-Related  Day  Care 
program 

Voluntary,  individualized  retention  and  advancement  program,  including  career  counseling  and 
service  referals;  provided  through  a welfare  agency  partnership  with  a nonprofit  employment 
service  provider 

Riverside  Post- 
Assistance  Self- 
Sufficiency 

Employed  former  TANF  recipients 

Voluntary,  marketed,  individualized  retention  and  advancement  program;  provided  primarily  by 
three  community-based  organizations  and  a community  college 

South  Carolina 

Individuals  who  left  TANF  for  any 
reason  between  October  1997  and 
December  2000 

Voluntary  program  offering  individualized  job  placement,  retention,  and  advancement  services 
with  modest  financial  incentives  for  program  engagement  and  employment;  provided  by  a welfare 
agency 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTES:  The  Corpus  Christi,  Fort  Worth,  and  Fiouston  sites  all  operated  the  Texas  ERA  program. 

ERA  programs  not  included  here  are  the  New  York  City  PRIDE  program,  the  Minnesota  program,  the  New  York  City  Substance  Abuse  Case 
Management  program,  and  the  Portland  (Oregon)  Career  Builders  program. 


The  History  of  ERA 


The  Problem:  Employment,  Wages,  and  Earnings  of  Single  Parents 

Due,  in  part,  to  the  1996  Personal  Responsibility  and  Work  Opportunity  Reconciliation 
Act  (PRWORA),  employment  rates  for  low-income  single  mothers  have  recently  risen.2  In  the 
early  1990s,  for  example,  the  employment  rate  for  low-income  single  mothers  was  about  67 
percent,  and  it  rose  to  78  percent  by  2000. 3 The  increase  was  especially  large  for  single  mothers 
who  had  received  welfare  in  the  previous  year  (and  may  have  remained  on  welfare),  for  whom 
rates  of  employment  doubled  over  this  time  period,  from  about  30  percent  in  the  early  1990s  to 
nearly  60  percent  in  2000.4 

The  wages  of  single  mothers,  however,  are  often  low.  Considering  all  workers  today, 
one  out  of  four  earn  less  than  $10  per  hour.5  While  some  of  these  low- wage  workers  are 
teenagers,  many  are  adults  supporting  families.  Considering  just  single  mothers,  the  25th 
percentile  wage  in  2000  was  $8.00  per  hour,  which  suggests  that  many  single  parents  are  not 
earning  a wage  that  is  high  enough  to  move  their  families  out  of  poverty.6  Moreover,  recent 
trends  in  wage  growth  over  time  do  not  bode  well  for  advancement.  While  women’s  wages 
have  not  stagnated  or  declined  as  much  as  men’s  wages,  from  2001  through  2006-2007,  wage 
growth  among  women  at  the  bottom  of  the  earnings  distribution  has  been  only  1 percent. 7 

In  addition,  job  retention  among  low- wage  workers  is  low.  Among  female  low-wage 
workers,  the  median  job  spell  duration  is  about  four  months,  and  about  four  in  ten  job  spells  end 
in  nonemployment.8 

Narrowing  the  group  of  interest  further,  to  welfare  leavers,  annual  earnings  and  income 
are  low:  average  earnings  are  below  $3,000  per  quarter  and  below  $10,000  per  year.9  This 
suggests  that,  despite  higher  employment  rates,  current  and  former  welfare  recipients  have  had 
trouble  maintaining  continuous  employment  and  advancing  in  the  labor  market. 


2Macroeconomic  conditions,  and  probably  the  increased  generosity  of  the  Earned  Income  Tax  Credit 
(EITC),  contributed  to  this  trend  as  well. 

3Holzer  and  Martinson  (2005). 

4Blank  and  Schmidt  (2001). 

5U.S.  Department  of  Labor,  Bureau  of  Labor  Statistics  (2008). 

6Lerman  (2005). 

7Mishel,  Bernstein,  and  Allegretto  (2007).  These  estimates  of  wage  growth  depend  on  the  type  of  inflation 
adjustment  that  one  uses.  In  addition,  the  estimates  do  not  take  into  account  the  rising  worth  of  health  insurance 
benefits  for  those  who  have  jobs  that  include  such  fringe  benefits. 

8Schochet  and  Rangarajan  (2004). 

9Acs  and  Loprest  (2004). 
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These  are  the  types  of  statistics  that  ERA  sought  to  improve,  by  identifying  and  testing 
a diverse  set  of  innovative  program  models  designed  to  promote  employment  stability  and 
advancement. 

Past  Efforts  to  Increase  Employment  Retention  and  Advancement  Among 

Low-Income  Individuals 

Research  on  several  different  types  of  policy  interventions,  described  below,  as  well  as 
the  past  program  experience  of  states  and  localities,  set  the  stage  for  the  ERA  project. 

Providing  Postemployment  Sen/ices  to  Newly  Employed  Welfare  Recipients 

The  Post-Employment  Services  Demonstration  (PESD)  was  a four-site  project  that  ex- 
amined the  effectiveness  of  providing  various  services  to  welfare  recipients  newly  employed  in 
part-time  or  full-time  jobs.  Operated  from  1994  to  1996,  the  programs’  offered  services  in- 
cluded counseling  and  support;  job  search  assistance;  resolution  of  government  benefits  issues; 
referrals  to  services  such  as  child  care,  training  or  education  programs,  legal  aid,  or  specialized 
counseling;  and  more  frequent,  larger,  and  flexible  payments  for  work-related  expenses.  The 
most  commonly  utilized  services  in  the  programs  were  counseling  and  work-expense  payments. 

The  PESD  programs  were  evaluated  using  a random  assigmnent  design,  where  control 
group  members  were  eligible  to  receive  minimal  case  management  services  provided  by  the 
welfare  department  for  a short  period  after  they  found  jobs  and  left  welfare.  A comparison  of 
the  labor  market  behavior  of  those  assigned  to  the  PESD  programs  with  the  behavior  of  those 
assigned  to  the  control  groups  showed  that  the  PESD  programs  had  little  effect  on  employment 
retention  or  earnings. 10 

Providing  Earnings  Supplements  or  Stipends  to  Low-Income  Individuals 

Several  random  assigmnent  studies  have  shown  that  supplementing  the  earnings  of  low- 
wage  workers  can  promote  employment  retention.  The  Minnesota  Family  Investment  Program 
(MFIP),  which  was  first  operated  in  1994,  allowed  welfare  recipients  to  keep  up  to  $250  more 
of  their  monthly  welfare  grant  when  they  went  to  work.  A study  of  two  versions  of  the  MFIP 
program  showed  that  both  versions  increased  overall  employment  levels  and  also  increased  the 
frequency  of  sustained  work.11  The  New  Hope  Project,  a demonstration  program  implemented 
in  two  imier-city  areas  in  Milwaukee  from  1994  through  1998,  offered  low-income  full-time 


10Rangarajan  and  Novak  (1999).  PESD  was  funded  by  the  Administration  for  Children  and  Families  with- 
in the  U.S.  Department  of  Health  and  Human  Services.  The  project  was  conducted  by  Mathematica  Policy 
Research. 

Gennetian.  Miller,  and  Smith  (2005). 
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workers  several  benefits:  an  earnings  supplement,  subsidized  health  insurance  and  child  care, 
and,  if  needed,  referrals  to  wage-paying  community  service  jobs.  New  Hope  increased  em- 
ployment as  well  as  earnings,  and  it  reduced  poverty  rates  as  well. 12  Canada’s  Self-Sufficiency 
Project  (SSP),  which  operated  in  two  provinces  as  a demonstration  project  from  1992  to  1999, 
offered  a monthly  earnings  supplement  to  single-parent  welfare  recipients  if  they  worked  full 
time.  This  program  increased  employment  levels  and  rates  of  employment  retention. 13,14 

Finally,  another  form  of  financial  incentives  — rent  breaks  for  public  housing  residents 
conditioned  on  work  — has  had  positive  effects  on  earnings  for  many  different  types  of  public 
housing  residents,  as  examined  in  the  Jobs-Plus  demonstration,  operated  from  1998  to  2003. 15 

Providing  Preemployment  Sen/ices  to  Welfare  Recipients 

Prior  to  PESD  and  ERA,  few  studied  welfare-to-work  programs  provided  any  formal 
postemployment  services  beyond  transitional  benefits.  Studies  of  welfare-to-work  programs  do 
suggest,  however,  that  preemployment  services  can  increase  stable  employment  and  earnings. 
In  the  National  Evaluation  of  Welfare-to-Work  Strategies  (NEWWS),  several  programs 
increased  individuals’  likelihood  of  remaining  employed  for  a year  or  more.  The  program  with 
the  largest  effect  on  stable  employment  and  earnings  growth  was  in  Portland,  Oregon.  While  the 
research  design  used  in  NEWWS  cannot  conclusively  indicate  why  these  effects  were  largest 
for  this  program,  the  Portland  program  was  unusual  in  that  it  allowed  some  individuals  to 
participate  in  short-term  training  or  education  before  they  searched  for  work  and  also  encour- 
aged participants  to  hold  out  for  jobs  that  paid  higher  than  the  minimum  wage  and  that  offered 
fringe  benefits. 16 

The  Development  of  the  ERA  Project 

Given  that  previously  tested  initiatives  that  focused  on  improving  retention  and  ad- 
vancement for  welfare  recipients  were  not  effective,  ERA  sought  to  examine  a variety  of 
innovative  models.  The  goal  was  to  determine  whether  any  strategies  could  be  effective,  given 
the  paucity  of  positive  results  in  this  area  in  the  past,  while  at  the  same  time  taking  account  of 


12Huston  et  al.  (2003). 

13Michalopoulos  et  al.  (2002). 

14More  recent  results  from  an  ongoing  study  of  an  ERA-like  trial  program  in  the  United  Kingdom  (UK) 
are  also  promising.  The  UK  Employment  Retention  and  Advancement  (ERA)  program  offered  a combination 
of  job  coaching  and  financial  incentives  tied  to  employment  retention  in  full-time  jobs  to  several  groups 
receiving  government  income  support.  Two-year  results,  based  on  a random  assignment  design,  are  showing 
that  the  program  has  increased  earnings  for  the  target  group  most  similar  to  the  groups  targeted  in  the  U.S. 
ERA  project  (Riccio  et  al.,  2008). 

15Bloom,  Riccio,  and  Verma  (2005). 

16Hamilton  (2002). 
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the  hypotheses  generated  by  the  studies  of  past  efforts.  Thus,  there  was  not  one  single  model 
tested  in  ERA.  Rather,  a wide  net  was  cast  in  which  to  identify  strategies  that  might  improve 
employment  retention  and  advancement  outcomes. 

The  project  began  in  1998,  when  the  Administration  for  Children  and  Families  (ACF) 
within  the  U.S.  Department  of  Health  and  Human  Services  (HHS)  issued  planning  grants  to  13 
states  to  develop  new  programs,  and  it  awarded  a contract  to  The  Lewin  Group  to  provide 
planning  assistance  to  these  states.  The  following  year,  ACF  selected  MDRC  to  conduct  an 
evaluation  of  the  ERA  models.17  From  2000  to  2003,  MDRC  and,  starting  in  2001  as  a subcon- 
tractor, The  Lewin  Group  worked  closely  with  the  states  that  had  received  planning  grants,  and 
with  several  other  states,  to  mount  tests  of  ERA  models.18  MDRC,  Lewin,  and  Cygnet  Asso- 
ciates also  provided  extensive  technical  assistance  to  some  of  the  states  and  program  operators, 
since  most  were  starting  from  scratch,  with  no  proven  models  on  which  to  build. 

For  the  ERA  evaluation,  ACF,  MDRC,  and  Lewin  focused  on  choosing,  and  assisting 
states  to  refine,  programs  that  would  test  two  different  (although  not  mutually  exclusive)  routes 
to  improved  economic  success.  One  type  of  program  sought  in  the  site  selection  process  focused 
on  increasing  individuals’  work  experience  and  on-the-job  training.  These  “work-based” 
programs  were  ones  that  assessed  individuals’  skills  and  interests,  focused  on  understanding 
employer  and  labor  market  needs,  and  tried  to  address  individuals’  barriers  to  advancement  and 
develop  individualized  career  path  plans  within  and  between  employers.  The  second  type  of 
program  sought  in  the  site  selection  process  emphasized  education  and  training.  These  “educa- 
tion/training-based” programs  were  attractive  because  many  low-income  individuals  have  low 
skill  levels,  and  it  was  thought  that  many  would  be  unlikely  to  remain  steadily  employed  or 
advance  to  better  jobs  unless  they  gained  additional  skills.  Thus,  programs  that  allowed  individ- 
uals to  combine  work  and  training  were  sought,  particularly  ones  that  gave  special  attention  to 
recruiting  participants,  made  accommodations  for  working  families,  tried  to  match  individuals 
and  programs,  and  encouraged  attendance  and  the  completion  of  classes.  Regardless  of  the  route 
to  economic  success  embodied  in  the  models,  it  was  understood  that  the  programs  would  call  on 
staff  to  acquire  new  skills,  to  address  a set  of  issues  different  from  those  that  they  were  used  to 
addressing,  and  to  perfonn  new  job  duties. 

The  program  models  that  eventually  were  selected  for  inclusion  in  the  ERA  project  — 
while  reflecting  the  above  two  possible  routes  to  improved  economic  success  — most  impor- 
tantly embodied  states’  and  localities’  choices  of  goals,  target  populations,  and  program 
features.  Moreover,  the  programs  were  paid  for  through  existing  funding  streams  (and  largely 


l7The  U.S.  Department  of  Labor  has  also  provided  funding  to  support  the  ERA  project. 

18The  Lewin  Group  worked  with  MDRC  on  site  development  prior  to  2001  as  an  independent  contractor 
as  well. 


not  through  special  demonstration  funding),  which  indicated  states’  commitment  to  a vision  of 
welfare  reform  that  included  a focus  on  the  long-term  self-sufficiency  of  families.  The  pro- 
grams, though  new,  were  designed  by  practitioners,  as  opposed  to  researchers,  and  were 
operated  in  real-world  conditions  and  not  in  “hothouses,”  or  protected  conditions. 

Ultimately,  a total  of  16  ERA  models  in  eight  states  were  included  in  the  ERA  project, 
with  multiple  tests  operating  in  some  states. 19  Many  of  the  tests  took  place  in  urban  areas, 
including  several  of  the  nation’s  largest  cities,  and  almost  all  were  relatively  large  in  scale, 
enrolling  over  1 ,000  people  over  a one-  to  three-year  period.  The  models  differed  as  to  whether 
their  most  emphasized  goal  was  employment  retention  or  advancement,20  but  almost  all  the 
models  served  current  or  former  recipients  of  Temporary  Assistance  for  Needy  Families 
(TANF),  the  cash  welfare  program  that  mainly  serves  single  mothers  and  their  children.  As 
described  in  the  next  section,  however,  the  models  varied  in  their  strategies  to  encourage 
employment  retention  and  advancement,  in  their  specific  target  groups,  and  in  their  program 
features. 


The  ERA  Models  Included  in  This  Report 

The  program  models  that  ultimately  were  studied  in  ERA  used  several  different  types  of 
strategies  to  facilitate  progress  along  work-based  or  education/training-based  routes  to  increase 
employment  retention  or  advancement.  As  mentioned  above,  this  particular  report  focuses  on  12 
of  the  16  ERA  models;  it  excludes  the  models  that  targeted  “harder-to-employ”  individuals. 
(Results  for  these  models  will  be  presented  in  other  reports.)  For  the  12  models  examined  in  this 
report,  this  section  first  highlights  three  of  the  major  strategies  used.  Next,  the  ERA  target 
groups  (or  served  populations)  are  described.  Finally,  the  section  discusses  the  various  types  of 
outcomes  that  were  sought  by  model  designers  and  program  operators. 

General  Strategies  of  the  ERA  Models  and  Their  Rationales 

The  strategies  adopted  by  states  and  localities  in  their  ERA  program  models  were  de- 
veloped in  accordance  with  program  operators’  views  of  the  primary  impediments  to  retention 


19Some  past  ERA  reports  listed  15  ERA  models.  The  total  increased  to  16  to  recognize  that  one  of  the  tests 
in  Riverside,  California,  actually  involved  two  models  with  different  sets  of  service  providers  and  program 
rules.  In  addition,  in  light  of  the  significant  differences  in  implementation  in  the  three  sites  that  operated  one  of 
the  16  models  (the  Texas  model),  effectiveness  estimates  for  the  Texas  model  are  presented  by  site.  Thus,  in 
total,  the  project  yields  18  independent  estimates  of  site  effectiveness,  where  “site  effectiveness”  refers  to  the 
effectiveness  of  different  models  or  to  the  effectiveness  of  a model  that  was  implemented  very  differently  in  a 
number  of  sites. 

20 A few  ERA  models  (not  included  in  this  report)  focused  on  an  early  stage  in  individuals’  employment 
careers  and  sought  to  increase  the  proportion  of  “harder-to-employ”  people  who  entered  the  labor  market. 
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and  advancement  faced  by  individuals  in  their  chosen  target  populations.  None  of  the  models, 
however,  attempted  to  address  labor  market,  or  demand-side,  issues.  They  did  not,  for  example, 
seek  to  affect  hiring,  training,  staffing,  or  advancement  in  particular  industry  sectors.  Rather,  all 
the  models  attempted  to  address  supply-side,  or  “worker-based,”  obstacles  to  economic  success. 

Within  the  broad  category  of  worker-based  strategies,  the  ERA  models  can  be  viewed 
as  embodying  three  types  of  strategies.  Some  models  sought  to  encourage  retention  and  ad- 
vancement by  focusing  on  these  goals  when  individuals  were  not  employed  but  were  seeking 
work;  some  models  delivered  services  in  new  ways;  and  some  models  provided  new  types  of 
services  or  new  combinations  of  services.  These  strategies  were  not  set  forth  at  the  outset  of  the 
ERA  project  as  site  selection  criteria;  rather,  these  strategies  are  presented  here  as  a way  of 
capturing  the  diversity  of  the  studied  ERA  models,  and,  in  fact,  any  one  ERA  model  often  was 
an  example  of  the  use  of  two  or  all  three  of  these  strategies.  Each  of  these  three  types  of  strate- 
gies and  their  rationales  are  described  below.  Finally,  the  models  were  intended  to  be  different 
from  those  studied  as  part  of  the  Post-Employment  Services  Demonstration  (PESD),  and  a brief 
description  of  how  the  ERA  and  PESD  models  differed  is  presented. 

Delivering  Sen/ices  to  Individuals  When  They  Were  Not  Employed 

As  shown  in  Table  1.2,  several  ERA  models  sought  to  increase  employment  retention 
and  advancement  by  providing  services  to  individuals  who  were  not  employed  but  looking 
for  jobs.  Designers  of  these  models  hoped  that  the  relationships  forged  between  program  staff 
and  individuals  while  they  were  searching  for  jobs  would  continue  after  individuals  found  jobs. 
In  the  Texas  ERA  model,  for  example,  staff  members  began  working  with  TANF  applicants 
and  recipients  while  they  were  seeking  employment  and  then  continued  to  work  with  individu- 
als after  they  found  jobs  and  even  after  they  left  the  TANF  rolls.  Designers  also,  in  some 
models,  sought  to  improve  employment  retention  and,  eventually,  advancement,  by  counseling 
unemployed  individuals  to  seek  jobs  that  were  in  — or  related  to  — their  fields  of  interest,  as 
opposed  to  seeking  any  type  of  job  during  a job  search.  This  was  based  on  evidence  from 
previous  research  on  preemployment  programs  that  suggested  that  the  message  delivered  in 
group  job  sessions  about  the  types  of  jobs  to  seek  could  affect  individuals’  employment  and 
earnings.  In  the  Los  Angeles  Enhanced  Job  Club  ERA  model,  for  example,  program  partici- 
pants were  urged  to  seek  their  most  desired  job  in  their  area  of  interest  during  a five- week  job 
club.  If  that  most  desired  job  was  not  attainable,  the  individual  could  “step  down”  the  search  to 
the  next  most  desired  job  in  that  field,  with  further  steps  “down”  if  jobs  were  not  secured.21 


21Designers  of  another  model  — the  South  Carolina  ERA  model  — concerned  about  sustained  poverty 
among  TANF  leavers,  sought  to  provide  services  to  another  “new”  target  group:  individuals  who  had  left 
TANF  up  to  three  years  earlier.  When  contacted  by  program  staff,  some  of  these  individuals  were  not  em- 

(continued) 
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Delivering  Services  in  New  Ways 

Many  of  the  ERA  models  used  different  types  of  innovative  organizational  features  to 
deliver  retention  or  advancement  services  in  new  ways.  While  TANF  agencies  almost  always 
had  lead  roles  in  implementing  the  ERA  models,  a number  of  models  provided  program 
services  through  partnerships  between  welfare  agency  staff  and  staff  from  other  organiza- 
tions — for  example,  community  colleges,  Workforce  Investment  Act  (WIA)  One-Stop 
contractors,  nonprofit  employment  service  providers,  and  community-based  organizations.  The 
rationales  for  using  these  service  delivery  arrangements  were  severalfold.  Some  model  design- 
ers expected  that  such  partnerships  would  leverage  resources.  Others  anticipated  that  the 
arrangements  would  bring  together  different  types  of  services  and  expertise  in  one  location  and, 
thus,  would  better  serve  program  participants.  In  the  Medford  ERA  model,  for  example,  a 
partnership  to  operate  the  program  was  developed  between  the  welfare  agency  and  a nonprofit 
employment  service  provider  that  was  a WIA  contractor.  The  partnership  was  intended  to 
leverage  this  contractor’s  expertise  and  workforce  development  network.  For  similar  reasons,  a 
community  college  that  also  operated  a WIA  One-Stop  Center  was  part  of  the  partnership 
operating  the  Eugene  ERA  model.  Finally,  other  model  designers  expected  that  nongovernmen- 
tal organizations  would  be  more  familiar  than  governmental  agencies  with  the  jobs  and  services 
available  in  a community  and,  fiirthennore,  that  individuals  would  be  more  likely  to  engage 
with  staff  from  nonwelfare  organizations  after  leaving  TANF.  This  was  another  motivation  for 
the  partnerships  developed  in  the  Medford  and  Eugene  ERA  models  and  also  came  into  play  in 
the  development  of  the  Riverside  PASS  ERA  model.  In  Riverside  PASS,  several  community- 
based  organizations  and  a community  college  were  chosen  to  provide  all  services. 

Many  ERA  models  included  increased  staff  flexibility,  for  staff  to  work  flexible  hours 
and  to  meet  with  program  participants  in  a variety  of  locations,  under  the  expectation  that  this 
would  make  it  easier  for  working  program  participants  to  interact  with  program  staff.  In  the 
Eugene  ERA  model,  for  example,  weekly  office  hours  were  held  at  a community  college.  Staff 
in  a number  of  programs  also  could  meet  with  participants  in  the  evenings  or  during  weekends. 

A few  ERA  models  sought  to  develop  linkages  with  employers.  Designers  of  the  Chi- 
cago ERA  model,  for  example,  contracted  with  a for-profit  company,  which  had  strong  relation- 
ships with  local  employers,  to  provide  all  ERA  services,  under  the  expectation  that  the  use  of 
this  provider  would  facilitate  the  identification  of  higher-paying  jobs  and  the  placement  of  ERA 
participants  into  those  jobs.  Staff  operating  the  Texas  ERA  model,  as  another  example,  con- 
ducted occasional  meetings  with  program  participants  at  their  workplace,  sometimes  meeting 
with  staff  at  the  employer’s  work  site  to  discuss  opportunities  for  advancement  that  existed  with 


ployed,  while  others  were  employed.  Services  provided  in  this  model  were  tailored  to  individuals’  specific 
situations  and  goals. 
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The  Employment  Retention  and  Advancement  Project 
Table  1.2 


ERA  Models:  Overview  of  Target  Populations, 

Selected  Ways  of  Delivering  Services,  and  Selected  Types  of  Services 


Target  Populations 


Not  employed  and 
receiving  TANF 

Employed  and 
receiving  TANF 

Employed  and 
not  receiving  TANF 
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Wavs  of  delivering  services 

Nongovernmental  organization  as  a provider  of  services  V ■</  V •/ 

V 

V V V V 

Flexible  staff  hours  and/or  locations  V V V 

V V V 

V V V 

Employer  linkages  ,/  y/ 

Tvnes  of  services 

V 

V 

Financial  work  incentives  V V V 

V 

V 

Supported  advancement  through  job  change  y y 

V 

V 

Education  and  training  referrals  and/or  incentives 

V V V V 

V V V V 

Staff  counseling  on  job-related  issues  V V V V •/ 

V V V V 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTES:  For  the  definitions  of  these  features,  see  Chapter  2.  For  discussion  of  the  full  set  of  features,  by  model,  see  Chapters  3,  4,  and  5. 

Check  marks  indicate  that  the  feature  is  present  in  the  ERA  model. 

The  models  and  their  implementation  often  evolved  over  the  study  period.  This  table  presents  the  features  experienced  by  the  majority  of  the  study 
participants  for  the  greatest  extent  of  time. 

The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  all  operated  the  Texas  ERA  model. 


that  employer.  Finally,  as  another  example,  designers  of  the  Cleveland  ERA  model  sought  to 
“take  the  services  to  the  workers”  and  situated  all  ERA  services  at  individuals’  places  of 
employment  (in  this  case,  long-tenn  nursing  care  facilities),  to  make  it  as  easy  and  convenient 
as  possible  for  program  participants  to  access  retention-related  services  and  to  engage  employ- 
ers in  the  effort.22 

Delivering  New  Types  of  Services  or  New  Combinations  of  Services 

The  ERA  models  provided  some  common  services,  most  of  which  have  been  offered  in 
previous  and  current  programs.  All  the  models,  for  example,  used  one-on-one  staff-client 
interactions  as  the  platform  from  which  program  services  were  delivered,  although  they  differed 
regarding  how  this  was  done,  that  is,  what  was  discussed  during  staff-client  interactions,  the 
intensity  of  the  interactions,  and  staff  configurations  and  roles.  In  addition,  job  search  assistance 
of  some  type  was  provided  in  almost  all  the  models,  to  initially  get  individuals  into  jobs,  to  get 
them  new  jobs  if  they  left  or  lost  a job,  and/or  to  find  people  better  jobs  while  they  were 
working.  In-depth  assessments  of  individuals’  needs,  interests,  and  employment  barriers  were 
also  done  in  many  sites,  and  referrals  to  mental  health  or  substance  abuse  treatment  or  counsel- 
ing could  be  made. 

All  the  ERA  models,  however,  experimented  with  providing  new  types  of  services  or 
combinations  of  services.  Several  models  provided  financial  incentives  or  rewards  connected 
to  employment  — of  a substantial  amount  in  one  model,  the  Texas  model.  Program  designers 
were  motivated  by  the  results  of  previous  studies  indicating  that  supplementing  the  earnings  of 
low-wage  workers  can  promote  employment  retention,  by  giving  people  additional  income  to 
meet  their  expenses  or  by  giving  them  greater  incentive  to  keep  working.  There  was  also  an 
expectation  that  this  feature  could  encourage  employment  entry  and  increase  individuals’ 
implicit  wage  levels.  In  the  Texas  ERA  model,  for  example,  program  participants  were  offered 
monthly  stipends  of  $200  for  each  month  in  which  they  worked  at  least  30  hours  per  week.  In 
the  South  Carolina  model,  participants  were  offered  $50  if  they  increased  their  wage  by  8 
percent  or  more,  moved  from  a part-time  job  to  a full-time  one,  or  moved  to  a job  with  benefits. 
Other  types  of  incentives  were  also  offered  in  some  models.  In  the  Chicago  model,  for  example, 
participants  were  offered  $75  transit  passes  if  they  returned  for  office  visits. 

Other  models  sought  to  promote  advancement  through  encouragement  to  change 
jobs.  In  these  models,  staff  encouraged  individuals  to  move  from  lower-paying  jobs  to  higher- 

~2While  the  Cleveland  ERA  model  might  appear  to  be  a “demand-side”  intervention,  given  the  location  of 
its  services  at  individuals’  workplaces,  the  model  did  not  have  any  components  that  sought  to  affect  hiring, 
staffing,  or  promotion  practices  at  the  workplace.  Only  one  model  component  — the  supervisory  training 
provided  to  staff  who  supervised  low-wage  workers  at  the  involved  employers  — attempted  to  improve  a 
workplace  practice. 
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paying  ones;  supported  individuals  when  they  made  such  changes;  and,  in  the  case  of  one  model 
(the  Chicago  ERA  model),  set  up  job  interviews  for  individuals  with  employers  who  had 
specific,  higher-paying  job  openings. 

Many  models  provided  encouragement  for  employed  individuals  to  participate  in 
education  and  training  and  offered  assistance  to  help  individuals  balance  education  or  training 
participation  with  employment.  Designers  of  these  models  viewed  skills  acquisition  as  a route  to 
sustained  employment  and  eventual  advancement,  but  they  believed  that  working  individuals 
needed  encouragement  and  support  to  engage  and  persist  in  education  or  training,  given  the 
difficultly  of  balancing  parenting,  work,  and  schooling.  The  Riverside  Training  Focused  model, 
for  example,  sought  to  improve  access  to  education  and  training  by  allowing  individuals  to 
replace  required  weekly  work  hours  with  hours  of  education  and  training  for  up  to  two  years,  by 
directing  participants  to  specific  education  and  training  providers  and  programs,  and  by  paying 
for  selected  education  and  training  slots,  if  needed. 

Finally,  many  of  the  models  provided  individuals  with  individualized  counseling  on 
job-related  issues,  under  the  expectation  that  this  could  both  prevent  unnecessary  job  loss  and 
educate  workers  on  how  to  advance  at  their  current  job. 

Departing  from  Programs  Studied  in  the  PESD  Project 

The  models  studied  under  the  ERA  project  also  were  designed  to  be  different  from  the 
ones  studied  in  the  Post-Employment  Services  Demonstration  (PESD),  given  the  general  lack  of 
economic  improvements  found  in  PESD.  The  ERA  models  departed  from  the  PESD  programs 
along  several  dimensions,  as  described  in  Box  1.1. 

ERA  Model  Target  Groups 

While  almost  all  the  models  targeted  current  or  former  recipients  of  TANF,  the  models 
differed  in  terms  of  when  services  were  first  provided,  and  to  whom.  Findings  for  the  ERA 
models  are  grouped  in  this  report  according  to  the  models’  target  populations,  that  is,  according 
to  when  staff  in  the  models  first  provided  services  to  individuals.  Separate  chapters  examine 
ERA  models  that  served  unemployed  TANF  recipients  (Chapter  3),  models  that  served  em- 
ployed TANF  recipients  (Chapter  4),  and  models  that  served  individuals  who  were  employed 
and  not  receiving  TANF  (Chapter  5). 

Presenting  the  results  in  these  three  target  groupings  has  several  advantages.  First, 
grouping  the  models  allows  practitioners  who  are  planning  to  implement  a program  for  a 
specific  target  population  to  easily  see  the  range  of  ERA  program  models  that  were  imple- 
mented for  that  population  and  also  the  models’  results.  Second,  given  that  the  models  in  each 
grouping  tend  to  have  similar  goals,  this  organization  of  models  facilitates  an  assessment  of 
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Box  1.1 


How  Did  the  ERA  Models  Differ  from  the  PESD  Programs? 

The  Post-Employment  Services  Demonstration  (PESD)  was  a four-site  project  that  ex- 
amined the  effectiveness  of  providing  case  management  to  newly  employed  welfare 
recipients.  The  project  used  a random  assignment  research  design.  Staff  in  the  1994  to 
1996  PESD  programs  were  able  to  contact  most  of  the  individuals  randomly  assigned  to 
the  program  group  and  offer  them  services.  The  most  common  services  utilized  in  the 
programs  were  counseling  and  work-expense  payments.  The  PESD  programs  generally 
failed,  however,  to  increase  employment  retention  or  earnings.  The  ERA  models  differed 
from  the  PESD  programs  in  several  ways: 

• ERA  had  greater  customization  of  services.  Most  of  the  ERA  models  sought  to  tailor 
services  to  particular  target  groups  and  often  had  divergent  goals  for  the  different  groups 
(for  example,  retention  for  newly  employed  individuals  and  advancement  for  those  steadi- 
ly employed).  In  contrast,  staff  in  the  PESD  programs  worked  only  with  newly  employed 
welfare  recipients,  and  services  were  aimed  primarily  at  increasing  retention. 

• ERA  staff  worked  with  individuals  who  were  not  employed.  In  about  half  the  ERA 
models,  program  staff  first  started  working  with  individuals  when  they  were  not  em- 
ployed. These  services  were  provided  to  establish  relationships  between  ERA  participants 
and  program  staff  prior  to  job  entry,  to  encourage  participants  to  start  thinking  about  reten- 
tion and  advancement  prior  to  taking  a job,  and  to  attempt  to  place  participants  in  better 
initial  jobs.  In  contrast,  PESD  staff  did  not  work  with  individuals  until  they  had  found 
jobs. 

• ERA  models  included  more  services,  additional  features.  As  examples,  some  ERA 
models  offered  financial  work  incentives  to  ERA  participants,  in  addition  to  other  services; 
other  models  sought  to  provide  more  opportunities  for  training;  one  model  had  staff  sta- 
tioned at  workplaces;  and  one  model  had  a job  search  approach  focused  on  individuals’ 
careers  of  interest. 

• ERA  had  greater  diversity  of  primary  service  providers.  Primary  service  providers  in 
the  ERA  models  included  organizations  such  as  community  colleges,  nonprofit  employ- 
ment agencies,  community-based  organizations,  and  for-profit  organizations  — in  addition 
to  welfare  agencies.  The  welfare  agency  often  partnered  with  these  other  organizations.  In 
the  PESD  programs,  staff  in  welfare  agencies  provided  all  services. 

• ERA  had  more  variation  in  service  delivery  methods.  ERA  models  experimented  with 
new  ways  to  deliver  services  — for  example,  by  organizing  staff  and  staff  roles  differently, 
placing  a greater  emphasis  on  career  counseling  and  coaching,  lowering  caseloads,  provid- 
ing more  proactive  follow-up,  increasing  the  availability  of  staff  to  individuals,  imple- 
menting more  comprehensive  assessments,  and  providing  a stronger  focus  on  education 
and  training. 

* 

Rangarajan  and  Novak  (1999). 
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how  well  similar  or  different  strategies  worked  in  achieving  similar  goals  for  roughly  similar 
populations.  Finally,  the  groupings  allow  one  to  see  whether  ERA  program  services  tend  to 
work  better  when  initiated  at  one  or  another  point  in  individuals’  employment  careers  — for 
example,  when  people  are  looking  for  work  or  when  they  are  already  in  work. 

It  should  be  kept  in  mind,  however,  that  many  ERA  sample  members  moved  in  and  out 
of  work  and  on  and  off  TANF  over  the  ERA  follow-up  period,  so  the  circumstances  of  the 
individuals  in  the  model  tests  are  not  static.  Staff  in  the  ERA  programs  in  each  of  the  three 
target  groupings  had  to  provide  different  types  of  services  over  time,  as  sample  members’ 
employment  and  welfare  situations  changed.  Furthermore,  changes  in  sample  members’ 
situations  did  not  always  go  in  one  direction.  While  many  unemployed  sample  members 
eventually  found  jobs,  a sizable  proportion  of  those  who  were  employed  as  of  study  entry  and 
who  worked  steadily  in  the  first  year  of  follow-up  did  not  work  or  worked  only  intermittently  in 
the  second  year  of  follow-up.  Similarly,  some  sample  members  who  were  not  TANF  recipients 
as  of  study  entry  became  TANF  recipients  at  some  point  during  the  follow-up  period.  Finally, 
states’  TANF  grant  levels  and  earnings  disregard  policies23  also  influenced  these  target  group- 
ings: some  employed  TANF  recipients  in  one  state  would  be  employed  TANF  leavers  in 
another  state. 

Sample  members’  changing  employment  and  welfare  situations  also  played  a role  in 
how  long  ERA  program  services  were  provided  to  individuals,  and  this  differed  by  model.  In 
some  models,  individuals  could  receive  services  as  long  as  they  had  a connection  with  TANF. 
In  other  models,  individuals  could  receive  services  for  a set  period  of  time  — for  example,  for 
one  year  in  one  model. 

Types  of  Retention  and  Advancement  Sought  in  ERA 

ERA  model  designers  sought  to  achieve  different  types  of  employment  retention  and 
advancement.  Regarding  retention,  the  models  in  many  cases  were  designed  to  encourage 
workers  to  stay  employed  in  the  job  they  held  as  of  staffs  first  contact  with  them,  with  staff 
counseling  individuals  regarding  on-the-job  issues  or  personal  or  family  problems  that  might 
interfere  with  their  job  perfonnance.  In  many  cases,  models  were  also  — or  instead  — designed 
to  encourage  workers  to  maximize  the  time  that  they  were  employed  in  any  job  and  not  neces- 
sarily in  the  job  they  held  as  of  program  staffs  first  contact  with  them  — through,  for  example, 
the  provision  of  help  to  quickly  find  another  job  following  a job  loss  or  the  provision  of  finan- 
cial incentives  tied  to  employment  (as  opposed  to  job)  retention.  Finally,  in  some  models  where 


~3The  earnings  disregard  is  the  amount  or  proportion  of  earnings  that  is  not  counted  when  calculating  the 
size  of  a welfare  grant.  The  earnings  disregard  creates  a greater  incentive  to  work  by  allowing  welfare 
recipients  to  keep  a large  portion  of  their  welfare  check  when  they  find  a job. 
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advancement  was  not  an  explicit  goal,  employment  retention  was  viewed  as  a long-term 
strategy  to  foster  advancement,  as  individuals  built  up  steady  work  experience  and  skills. 

In  most  cases,  model  designers  defined  advancement  broadly,  as  increased  earnings.  In 
these  cases,  longer  employment  spells  with  fewer  gaps  between  them,  increases  in  work  hours, 
and  wage  increases  would  all  contribute  to  increased  earnings.  Operators  of  at  least  one  model 
viewed  advancement  as  movement  from  informal  jobs  with  few  benefits  to  more  formal  jobs 
with  more  benefits,  particularly  if  there  were  wage  increases.  More  fonnal  jobs  also  increased 
the  opportunity  for  individuals  to  become  eligible  for  unemployment  insurance  benefits,  Social 
Security  benefits,  and  Medicare.  Advancement  also  involved  seeking  to  place  individuals  who 
were  working  in  fonnal  jobs  into  better,  more  highly-paid  jobs.  In  addition,  the  provision  of 
help  to  quickly  find  another  job  following  a job  loss  was  seen  in  some  models  as  a way  not  only 
to  increase  employment  retention  but  also  as  a way  to  foster  advancement,  if  a better  job  than 
the  last  job  could  be  obtained.  Finally,  improvements  in  fringe  benefits  received,  job  stability, 
working  conditions,  or  opportunities  for  promotion  were  also  often  viewed  as  aspects  of 
advancement. 

Sometimes  “retention”  and  “advancement”  were  complementary,  as  when  employment 
retention  allowed  an  individual  to  acquire  the  work  experience,  job  skills,  and  seniority  that  led 
to  a promotion  or  a better  resume  for  a prospective  new  employer.  Sometimes,  however,  they 
were  at  odds,  as  when  an  individual  would  forgo  employment  in  order  to  participate  full  time  in 
education  or  training. 


The  ERA  Research  Design 

Each  of  the  ERA  models  discussed  in  this  report  was  studied  using  a random  assign- 
ment evaluation  design  — a methodology  that  allows  practitioners  and  policymakers  to  have  a 
high  degree  of  confidence  in  the  results.  In  each  site,  individuals  who  met  the  ERA  eligibility 
criteria  (which  varied  by  site)  were  assigned,  at  random,  to  a program  group,  usually  called  “the 
ERA  group,”  or  to  a control  group.  Members  of  the  program  group  were  recruited  for  (and,  in 
some  sites,  were  required  to  participate  in)  the  services  offered  by  the  ERA  program,  while 
those  in  the  control  group  were  not  eligible  for  ERA  services.  Both  groups  were  eligible  for  the 
services  and  supports  generally  available  in  the  site’s  community  and,  often,  for  the  site’s 
standard  welfare-to-work  program  or,  in  some  cases,  minimal  efforts  that  the  site  already  had  in 
place  to  provide  assistance  to  individuals  who  found  jobs.  Both  groups  also  were  eligible  for  the 
child  care  and  transportation  assistance  that  is  normally  provided  to  TANF  recipients  or  to 
working  TANF  leavers,  either  to  support  participation  in  employment-promoting  activities  or  in 
employment  itself.  The  control  groups  thus  were  not  “no  service”  groups.  Rather,  each  site’s 
control  group  is  the  benchmark  against  which  that  site’s  ERA  approach  is  assessed,  and  the 
employment  retention  and  advancement  outcomes  of  the  control  group  over  time  represent  what 
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would  have  happened  in  the  absence  of  the  site’s  ERA  program.  Comparisons  between  the 
program  group  and  the  control  group  thus  represent  the  “added  value”  of  the  ERA  services.24 
Table  1.3  briefly  summarizes  the  program-control  treatment  comparisons  examined  in  each  of 
the  ERA  tests  discussed  in  this  report. 

Sample  Sizes  and  Follow-Up  Periods 

In  total,  over  45,000  individuals  were  randomly  assigned  to  research  groups  as  part  of 
ERA,  starting  in  2000  in  the  earliest-starting  test  and  ending  in  2004  in  the  latest-starting  test.25 
In  each  site,  MDRC  tracked  individuals  in  both  research  groups  for  three  to  four  years  following 
their  random  assignment,  using  administrative  records  — such  as  unemployment  insurance 
records,  welfare  records,  and  food  stamp  records  — and  sample  member  surveys.  The  random 
assignment  process  ensured  that  when  individuals  entered  the  study,  there  were  no  systematic 
differences  in  sample  members’  characteristics,  measured  or  umneasured,  between  the  research 
groups.  Thus,  any  differences  between  them  that  emerge  after  random  assigmnent  (for  example, 
in  employment  rates  or  average  earnings)  can  be  attributed  to  the  site’s  ERA  model  — in 
contrast  to  services  and  supports  already  available  in  the  sites. 

The  Definition  of  “Impacts” 

Differences  in  outcomes  between  the  research  groups  after  random  assigmnent  are 
known  as  “impacts,”  and  they  indicate  the  effects  of  the  ERA  program  in  each  site.  Tests  of 
statistical  significance  were  performed  on  all  impacts  presented  in  the  report,  to  determine 
whether  an  impact  could  confidently  be  attributed  to  an  ERA  program.  Except  where  otherwise 
indicated,  only  statistically  significant  impacts  are  discussed  in  the  report.26 

The  above-described  experimental  (random  assignment)  design  enables  researchers  to 
attribute  impacts  to  the  effects  of  the  studied  ERA  models  with  a high  degree  of  confidence. 
Each  ERA  model,  however,  offered  a package  of  services  and  supports  to  individuals.  The  type 
of  random  assigmnent  design  that  was  implemented  in  ERA  does  not  support  the  parsing  out  of 
the  relative  contributions  of  specific  program  design  or  implementation  features  to  the  models’ 
economic  impacts. 


24  Across  the  sites,  the  control  group  benchmark  differed,  in  terms  of  the  types  of  services  for  which  con- 
trol group  members  were  eligible,  the  extent  of  services,  and  how  well  known  and  readily  accessible  the 
services  were.  None  of  the  control  groups,  however,  can  be  strictly  considered  to  be  a “no  service”  group. 

25Outcomes  for  approximately  27,000  sample  members  are  analyzed  in  this  report.  See  Chapter  2 for  a 
discussion  of  the  other  documents  in  which  results  for  the  remaining  sample  members  will  be  presented. 

~6An  impact  is  considered  statistically  significant  if  there  is  less  than  a 10  percent  probability  that  the  esti- 
mated difference  would  have  occurred  by  chance  in  the  absence  of  any  effect  of  the  program.  The  report’s 
tables  and  figures  note  statistical  significance  at  the  10  percent,  5 percent,  and  1 percent  levels. 
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The  Employment  Retention  and  Advancement  Project 
Table  1.3 

Summary  of  Comparisons  Examined  in  Each  ERA  Test 


Program 

Program  Group  Treatment 

Control  Group  Treatment 

Texas 

Corpus  Christi 
Fort  Worth 
Flouston 

Mandatory  preemployment  job  search  assistance  to  continue  to 
recieve  TANF  grant,  followed  by  voluntary  postemployment 
assistance  (which  could  include  employer  site  visits  and 
reemployment  assistance),  with  a monthly  stipend  of  $200  for 
former  TANF  recipients  working  at  least  30  hours  per  week; 
services  provided  by  staff  in  nonprofit  organizations  contracted 
by  local  workforce  agencies 

Mandatory  (for  TANF  receipt)  preemployment  job  search 
assistance,  followed  by  limited,  voluntary  postemployment 
assistance  with  no  stipend;  services  provided  by  staff  in  nonprofit 
organizations  contracted  by  local  workforce  agencies 

Los  Angeles  EJC 

Mandatory  (for  TANF  receipt)  5-week  job  club  focused  on 
career  development  activities  and  job  search  targeted  to 
individuals’  careers  of  interest;  operated  jointly  by  county 
welfare  and  education  agency  staff 

Mandatory  (for  TANF  receipt),  traditional  3-week  job  club 
focused  on  getting  any  type  of  job  quickly;  operated  jointly  by 
county  welfare  and  education  agency  staff 

Salem 

Mandatory  (for  TANF  receipt)  preemployment  job  search 
assistance  and  voluntary  postemployment  services;  jointly 
provided  by  welfare  agency  and  community  college  staff 
located  at  the  WIA  One-Stop  Career  Center 

Mandatory  (for  TANF  receipt)  preemployment  job  search 
assistance  services;  jointly  provided  by  welfare  agency  and 
community  college  staff  located  at  the  welfare  office 

Chicago 

Mandatory  (in  addition  to  existing  TANF  requirements),  work- 
focused  advancement  program  offering  targeted  job  search 
assistance  and  help  to  identify  and  access  career  ladders; 
provided  by  staff  in  a private,  for-profit  firm. 

Mandatory  (standard  TANF  requirements)  less  intensive  and 
more  retention-oriented  program;  provided  by  staff  at  the  local 
welfare  agency 

(continued) 


Table  1.3  (continued) 


Program 

Program  Group  Treatment 

Control  Group  Treatment 

Los  Angeles  RFS 

Voluntary  (beyond  TANF  work  requirements),  intensely 
marketed,  individualized  retention  and  advancement  program; 
administered  by  welfare  agency  staff 

Voluntary  (beyond  TANF  work  requirements),  less 
individualized,  and  more  “rule-bound”  postemployment  program; 
administered  by  welfare  agency  staff 

Riverside  Training 
Focused 

Education/training-focused  advancement  program  with 
flexibility  to  reduce  or  eliminate  required  work  hours  (for 
TANF  receipt)  if  participating  in  education  or  training; 
operated  by  workforce  agency 

Work-focused  advancement  program  with  no  flexibility  to  reduce 
or  eliminate  required  work  hours  (for  TANF  receipt)  if 
participating  in  education  or  training;  operated  by  county  welfare 
agency 

Riverside  Work 
Plus 

Education/training-focused  advancement  program  with  less 
flexibility  (compared  to  the  Riverside  Training  Focused 
program)  to  reduce  or  eliminate  required  work  hours  (for 
TANF  receipt)  if  participating  in  education  or  training; 
operated  by  county  welfare  agency 

Work-focused  advancement  program  with  no  flexibility  to  reduce 
or  eliminate  required  work  hours  (for  TANF  receipt)  if 
participating  in  education  or  training;  operated  by  county  welfare 
agency 

Cleveland 

Voluntary  (with  active  recruitment),  employer-based 
employment  retention  program,  including  ongoing  staff-client 
relationships,  weekly  peer  support  groups,  and  supervisory 
trainings;  provided  by  a community-based  organization 

Voluntary  (intermittently  offered)  employer-based  counseling, 
should  individuals  choose  to  pursue  it 

Eugene 

Voluntary,  individualized  program,  including  career 
counseling  and  service  referals;  provided  through  a welfare 
agency  partnership  with  a community  college  and  WIA 
contractor 

Voluntary,  nonindividualized  postemployment  services  normally 
offered  in  the  community,  should  individuals  choose  to  pursue 
them 

Medford 

Voluntary  (with  active  recruitment),  individualized  retention 
and  advancement  program,  including  career  counseling  and 
service  referrals,  provided  through  a welfare  agency 
partnership  with  a nonprofit  employment  service  provider 

Voluntary,  nonindividualized  postemployment  services  normally 
offered  in  the  community,  should  individuals  choose  to  pursue 
them 

(continued) 


Table  1.3  (continued) 


Program 

Program  Group  Treatment 

Control  Group  Treatment 

Riverside  PASS 

Voluntary,  marketed,  individualized  retention  and 
advancement  program;  provided  primarily  by  three  community- 
based  organizations  and  a community  college 

Voluntary,  less-intensive  postemployment  services  provided  by 
staff  in  local  welfare  agency,  should  individuals  choose  to  pursue 
them 

South  Carolina 

Voluntary  program  (with  active  recruitment)  offering 
individualized  job  placement,  retention,  and  advancement 
services  with  modest  financial  incentives  for  program 
engagement  and  employment;  provided  by  a welfare  agency 

Voluntary  employment-related  services  normally  available  in  the 
community,  should  individuals  choose  to  pursue  them 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTE:  The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  all  operated  the  Texas  ERA  model. 


In  a few  places  in  this  report,  comparisons  are  reported  that  are  not  experimental.  Most 
such  “nonexperimental  analyses”  examine  earnings  only  among  sample  members  who  were 
employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be 
attributable  to  the  ERA  program.  Statistical  tests  are  not  perfonned  for  these  analyses.  The  text 
and  tables  make  it  clear  when  comparisons  are  nonexperimental. 

As  a means  of  advancing  the  state  of  knowledge  in  this  field  about  what  types  of  inter- 
ventions should  be  studied  further  in  the  future  — and  to  provide  guidance  to  practitioners  about 
how  programs  that  aim  to  promote  employment  retention  and  advancement  among  low-wage 
workers  might  best  be  designed  and  operated  — systematic  nonexperimental  analyses  are  also 
being  undertaken  as  part  of  ERA.  These  analyses  will  be  described  in  future  ERA  documents. 

The  Implications  of  When  Random  Assignment  Occurred 

Random  assignment  occurred  at  different  points  in  the  various  ERA  tests,  but  broad 
patterns  exist  within  each  of  the  three  target  groupings.  In  general,  individuals  were  randomly 
assigned  without  regard  to  whether  they  were  interested  in  receiving  retention/advancement 
services;  that  is,  individuals  usually  were  automatically  included  in  the  study  and  in  the  random 
assignment  process  if  they  met  a site’s  eligibility  criteria.  But,  after  random  assignment  oc- 
curred, in  most  models  those  who  were  assigned  to  the  program  group  were  not  required  to 
participate  in  retention/advancement  services.  While  in  some  models  individuals  were  required 
after  random  assignment  to  look  for  jobs  if  they  were  unemployed,  in  most  models,  individuals’ 
participation  in  retention/advancement  services  — that  is,  the  ERA  services  that  were  generally 
added  to  preexisting  programs  — was  voluntary.  As  a result,  program  staff  usually  needed  to 
market  ERA  services  to  those  who  were  randomly  assigned  to  the  program  group;  individuals 
included  in  the  samples  were  not  necessarily  interested  in  receiving  such  services,  required  to 
participate  in  them,  or  motivated  to  seek  them  out  on  their  own  initiative.  Levels  of  service 
receipt  are  important  in  a random  assignment  study,  since,  as  discussed  below,  to  the  extent  that 
participation  in  services  is  low  in  both  the  program  and  the  control  group,  the  program-control 
difference  in  service  receipt  will  be  small,  lowering  the  probability  that  the  models  will  produce 
positive  impacts  on  employment  or  earnings. 

For  models  that  targeted  unemployed  TANF  recipients,  random  assigmnent  oc- 
curred when  individuals  applied  for  TANF  or  were  detennined  to  be  eligible  to  continue  to 
receive  TANF.  In  these  model  tests,  persons  randomly  assigned  to  the  program  groups  were 
required  to  participate  in  activities  aimed  at  placing  them  in  jobs,  and  many  of  them  did  so;  once 
they  were  employed,  participation  in  ERA  and  other  services  was  generally  voluntary,  and 
participation  in  ERA  services  was  not  as  universal. 
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For  models  that  targeted  employed  TANF  recipients,  random  assigmnent  generally 
occurred  when  welfare  agency  staff  received  notice  that  a welfare  recipient  had  found  a job  or 
had  held  a job  for  a certain  period  of  time  while  receiving  TANF.  In  most  of  these  cases, 
individuals  who  were  randomly  assigned  to  the  program  groups  were  required  to  work  or 
participate  in  work-related  activities  for  a certain  number  of  hours  each  week,  in  line  with 
general  TANF  requirements,  but  they  were  not  necessarily  required  to  participate  in  ERA 
activities. 

For  models  that  targeted  individuals  who  were  employed  and  not  receiving  TANF, 

random  assignment  occurred  at  a different  point  in  each  model  test.  In  several  of  the  tests, 
individuals  were  randomly  assigned  when  welfare  agency  staff  received  notice  that  they  had 
become  ineligible  for  TANF  due  to  earnings  above  a certain  level.  In  two  of  these  tests,  random 
assigmnent  was  done  regardless  of  individuals’  interest  in  services;  in  one  of  these  tests, 
individuals  were  asked  whether  they  were  interested  in  receiving  postemployment  services,  and, 
if  so,  they  were  randomly  assigned.  In  one  of  the  other  tests  in  this  grouping,  TANF  leavers 
were  randomly  assigned  regardless  of  their  current  employment  status  or  interest  in  receiving 
ERA  services.  In  the  final  test  in  this  grouping,  low-wage  workers  at  long-term  nursing  care 
facilities  who  were  interested  in  receiving  ERA-type  services  were  randomly  assigned.  Thus, 
while  some  sample  members  in  this  final  grouping  of  tests  were  likely  interested  in  receiving 
ERA-like  services  and,  if  placed  in  a control  group,  may  have  sought  such  services  out  on  their 
own,  this  was  not  necessarily  the  case  for  the  vast  majority. 


Economic  Environments  and  Service  Settings  of  the  ERA  Tests 

As  discussed  at  the  outset  of  this  chapter,  broad  economic  trends  have  elevated  the  im- 
portance of  steady  employment  and  eventual  wage  advances  as  means  to  economic  well-being 
and  self-sufficiency  for  low-wage  workers.  In  addition,  the  current  economic  climate  has 
increased  the  number  of  low-wage  workers  at  risk  of  losing  jobs  and  advancement  opportuni- 
ties. This  section  discusses  the  economic  environments  of  the  ERA  programs’  operation  and 
follow-up  periods  (roughly  2001  through  2007),  as  well  as  the  services  that  would  have  been 
available  to  control  group  members  during  this  time  period. 

Economic  Environments 

The  ERA  programs  operated  in  a fairly  challenging  labor  market  context.  The  United 
States  economy  was  in  a recessionary  state  for  most  of  2001.  The  unemployment  rate  reached  a 
high  of  6 percent  in  2003,  and  employment  (that  is,  the  number  of  jobs  reported  by  employers) 
did  not  return  to  prerecession  levels  until  2005.  This  fairly  long  postrecession  labor  market 
slump  — often  referred  to  as  a “jobless  recovery”  due  to  the  lack  of  labor  market  gains  despite 
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increases  in  productivity  and  other  macroeconomic  indicators  — was  particularly  difficult  for 
low- wage  workers.27 

There  was  also  considerable  regional  variation  in  labor  market  conditions  around  the 
country  during  the  period  under  study  in  ERA.  In  one  ERA  site,  for  example,  there  was  job 
growth  in  every  year  between  2000  and  2007,  and  most  of  the  unemployment  was  due  to 
workers’  transitioning  into  the  labor  market  and  between  jobs.  In  contrast,  as  another  example, 
another  site’s  unemployment  rate  remained  high,  and  the  number  of  jobs  in  the  area  steadily 
declined. 

The  random  assignment  evaluation  design  equalizes  the  effect  of  varying  labor  market 
conditions  between  the  control  and  program  groups.  That  is,  whether  the  site  had  an  expanding, 
stagnant,  or  contracting  labor  market  is  “controlled  for”  by  the  design  and  so  is  not  a factor  in 
the  size,  direction,  and/or  existence  of  differences  in  the  outcomes  of  program  group  members 
compared  with  control  group  members.  Thus,  the  economic  environments  surrounding  the  ERA 
tests  do  not  explain  the  programs’  impacts.  Placing  the  impacts  within  a site’s  labor  market 
context,  however,  can  assist  in  understanding  why  a program  produced  the  observed  impacts.28 

Preexisting  Services  and  Supports 

Another  environmental  factor  was  the  preexisting  services  and  supports  available  in  the 
sites’  communities.  In  many  sites,  as  is  discussed  in  the  report,  control  group  members  partici- 
pated in  services  at  significant  levels.  This  is  likely  due  to  two  developments.  First,  the  services 
and  supports  offered  to  control  group  members  as  part  of  the  sites’  “standard”  welfare-to-work 
programs  reflected  localities’  responses  to  the  many  welfare  reform  initiatives  put  into  place 
since  the  late  1980s  and,  possibly,  to  research  findings  about  effective  approaches.  These 
initiatives  included  the  1988  and  1996  welfare  reform  acts  that  changed  the  conditions  of 
welfare  and  the  1998  reform  of  the  workforce  development  system.  The  services  and  supports 
available  to  control  group  members  also  reflected  changes  in  welfare  earnings  disregard  policies 
and  expansions  in  the  Food  Stamp  Program,  child  care  assistance,  and  the  Earned  Income  Tax 
Credit  (EITC)  that  have  increasingly  supported  low-income  individuals  who  work.  Overall, 
these  changes  have  resulted  in  more  services  and  supports  being  offered  to  low-wage  workers. 
Second,  control  group  members  in  the  ERA  study  often  had  access  to  services  in  their  greater 
environments,  provided  by  community  colleges,  workforce  agencies,  and  other  organizations. 


27Economic  Policy  Institute  (2006). 

28For  each  site  discussed  in  this  report,  a short  summary  of  the  local  economic  conditions  is  included.  This 
summary  provides  information  as  to  the  conditions,  but  it  does  not  attempt  to  explain  “why”  the  local  econom- 
ic conditions  in  any  given  site  are  relatively  good  or  bad. 
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These  services  would  have  been  in  addition  to  what  the  sites’  welfare  agencies  would  have 
offered  as  part  of  their  welfare-to-work  programs  or  minimal  postemployment  services. 


Characteristics  of  ERA  Sample  Members  and  the  Labor  Market 
Behavior  of  Control  Group  Members 

Across  the  sites,  sample  members  in  the  ERA  tests  were  primarily  female  single  parents 
who  were  fonner  or  current  welfare  recipients.29  At  study  entry,  the  average  sample  member 
had  more  than  one  child,  and  almost  two-thirds  had  at  least  one  child  who  was  an  infant  or 
toddler.  Approximately  half  the  sample  members  did  not  have  a high  school  diploma,  and  more 
than  a third  had  a welfare  receipt  history  of  two  years  or  longer.  Among  those  for  whom  an 
employment  history  was  available,  over  90  percent  had  had  some  employment  in  the  past  three 
years,  but  less  than  a third  had  worked  for  more  than  24  months  in  that  period.  Considering 
ERA  sample  members  who  were  employed  as  of  random  assignment,  less  than  half  worked 
full-time  hours.30  While  the  characteristics  of  sample  members  varied  across  the  sites,  many 
sample  members  at  each  site  faced  multiple  obstacles  to  finding  and/or  keeping  employment.31 

As  discussed  above,  retention  of  employment  and  advancement  in  the  labor  market  has 
been  difficult  for  low-wage  workers.  The  employment  patterns  of  the  ERA  control  group 
members,  from  their  study  enrollment  through  the  follow-up  period  covered  in  this  report, 
illustrate  these  issues.  In  ERA  site  tests  targeting  employed  current  or  fonner  welfare  recipients, 
unemployment  insurance  data,  which  are  available  in  quarterly  “chunks,”  indicate  that  only  half 
the  control  group  members  who  were  employed  in  the  quarter  following  random  assignment 
were  employed  continuously  for  two  years.  Switching  the  focus  to  the  first  job  (employer)  after 
random  assigmnent,  average  job  tenure  in  site  tests  targeting  nonemployed  recipients  ranged 
from  1.4  quarters  to  1.7  quarters,  or  fewer  than  six  months;  in  site  tests  targeting  employed 
recipients,  these  figures  were  higher,  with  average  job  tenure  ranging  from  3.7  quarters  to  5.6 
quarters. 


29The  Cleveland  ERA  test’s  sample  was  distinct  from  the  samples  in  the  other  sites.  Cleveland  sample 
members  had  little  or  no  recent  connection  to  public  assistance,  and  they  were  not  necessarily  single  parents. 

30The  Cleveland  sample  is  an  exception  to  this  case  as  well;  over  80  percent  of  Cleveland  sample  mem- 
bers were  hill-time  workers. 

3 'ERA  sample  members,  most  of  whom  were  low-wage  workers,  represent  a particular  subset  of  the  gen- 
eral population  of  low- wage  workers.  Nationally,  about  half  of  all  low-wage  workers  are  female,  and  only  a 
fifth  are  in  single-parent  households.  In  contrast  to  the  ERA  sample,  only  a minority  of  low-wage  workers 
nationally  do  not  have  at  least  a high  school  diploma  or  equivalent  credential,  and  almost  three-quarters  of 
them  work  full-time  hours.  (These  characteristics  of  the  national  population  of  low- wage  workers  are  based  on 
calculations  from  the  March  2005  Current  Population  Survey  [U.S.  Census  Bureau,  2005].) 
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Moreover,  job  loss  was  often  very  quick.  Data  from  the  ERA  12-Month  Survey  suggest 
that  about  half  of  sample  members  in  the  model  tests  targeting  employed  individuals  had  either 
already  become  unemployed  during  the  period  between  being  identified  as  eligible  for  ERA  and 
being  randomly  assigned  or  had  left  the  job  that  they  held  as  of  random  assignment  within  90 
days. 


The  difficulties  in  remaining  employed  are  reflected  in  the  modest  earnings  changes  ob- 
served for  control  group  members.  Between  the  first  and  second  years  after  random  assigmnent, 
their  average  annual  earnings  increased  by  about  $500.  However,  only  a third  of  the  control 
group  members  had  higher  earnings  in  the  second  year  than  in  the  first,  and  about  half  of  those 
with  an  earnings  gain  experienced  an  improvement  of  less  than  $1,000  per  quarter.  As  a further 
indication  of  employment  instability,  no  earnings  change  statistic  over  the  first  two  years  of 
follow-up  could  be  calculated  for  43  percent  of  the  control  group  members  because  they  were 
not  employed  in  one  or  both  years. 

While  the  ERA  sample  members  represent  a particular  subset  of  the  low-wage  worker 
population,  their  retention  and  advancement  difficulties  are  typical  of  the  broader  population. 
Small  proportions  of  low- wage  workers  are  able  to  remain  consistently  employed,  increase  their 
earnings,  and  move  up  in  the  workforce  pennanently;  most,  however,  do  not.32 


Research  Questions  Addressed  in  This  Report 

In  brief,  this  report  addresses  the  following  major  questions. 

• Implementation.  What  does  it  take  to  mount  these  types  of  programs?  What 
methods  did  program  staff  use  to  engage  targeted  individuals?  What  services 
were  delivered?  To  what  extent  did  the  ERA  programs  increase  engagement 
and  deliver  different  types  of  services,  relative  to  what  normally  would  hap- 
pen? 

• Impacts.  To  what  extent  did  the  ERA  models  improve  employment,  em- 
ployment retention,  and  advancement  beyond  the  levels  attained  by  control 
group  members,  over  a three-  to  four-year  follow-up  period? 

Additional  questions  relating  to  the  costs  and  benefits  of  selected  ERA  models  are  ad- 
dressed in  a companion  report.33  That  document  examines  such  questions  as:  To  what  extent  did 
selected  ERA  programs  produce  gains  or  losses  from  different  perspectives  — those  of  the 


32See  Theodos  and  Bednarzik  (2006);  Gottschalk  (1982);  Smith  and  Vavrichek  (1992);  Swaim  (1997). 
33Redcross,  Deitch,  and  Farrell  (2010). 
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participants,  overall  government  budgets,  and  society  as  a whole?  What  was  the  cost- 
effectiveness  of  these  programs? 

Findings  from  all  the  models  in  the  ERA  project  contribute  to  addressing  all  the  above 
questions.  All  the  sites  involved  in  ERA  stepped  forward  to  innovate  in  a challenging  and 
important  area  of  social  policy  and  practice,  and  knowledge  can  be  gained  from  models  that  did 
not  show  economic  impacts  as  well  as  from  those  that  did. 


Considerations  in  Interpreting  ERA  Impacts 

As  discussed  above,  states  and  localities  used  a number  of  different  strategies  in  their 
program  models  to  promote  employment  retention  and  advancement.  When  actually  put  into 
the  field,  however,  the  implementation  of  the  ERA  models  often  differed  in  some  ways  from  the 
models  as  they  were  designed.  In  some  cases,  for  example,  despite  training  provided  by  MDRC 
and  ERA  project  consultants,  program  staff  did  not  have  the  experience  or  skills  needed  to  fully 
provide  the  types  of  services  envisioned  in  a model.  In  a few  cases,  as  another  example,  funding 
cuts  forced  alterations  in  the  model  designs  toward  the  end  of  their  implementation  periods.  34 

In  addition,  take-up  of  services  by  program  group  members  sometimes  varied  from  the 
levels  originally  expected,  and  this  often  differed  by  type  of  service  or  activity.  In  models 
targeted  to  TANF  recipients,  participation  in  activities  such  as  job  search  assistance  or  even 
education  or  training  courses  was  often  fairly  high,  as  these  activities  were  often  mandated  as 
part  of  individuals’  TANF  quid  pro  quo.  Even  among  individuals  not  receiving  TANF,  program 
group  members’  participation  rates  in  these  activities  were  sometimes  substantial.  In  contrast, 
program  group  members’  participation  in  other  types  of  activities  (such  as  receiving  counseling 
or  help  relating  to  how  to  deal  with  problems  on  a job,  assess  one’s  career  aptitudes  and 
possible  options,  or  find  a better  job  while  working)  was  often  low  — among  those  both  on  and 
off  the  TANF  rolls. 

In  a random  assignment  evaluation,  however,  the  activities  of  the  control  group  are  as 
important  to  take  into  account  as  those  of  the  program  group.  In  many  of  the  ERA  models 
examined  in  this  report,  the  control  group  levels  of  participation  in  some  activities  were  higher 
than  expected.  But  control  group  levels  of  participation  — and  whether  or  not  the  levels  were 
“unexpected”  — differed  by  the  type  of  service  or  activity. 


34ln  fact,  a test  of  an  ERA  model  initially  included  in  the  ERA  project  — operated  in  Memphis,  Tennessee 
— was  dropped  from  the  project  early  on,  due  to  the  site’s  inability  to  implement  the  model.  In  addition, 
evaluation  activities  for  one  of  the  16  tests  included  in  ERA  — a test  of  the  Portland,  Oregon,  Career  Builders 
model,  which  served  harder-to-employ  individuals  — eventually  were  curtailed  after  it  became  clear  that  the 
model’s  innovative  features  could  not  be  implemented. 
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As  a result,  program-control  differences  in  service  receipt  or  activity  participation  va- 
ried by  model  as  well  as  by  type  of  service  or  activity.  In  some  cases,  these  differences  were 
small,  due  to  low  rates  of  receipt  or  participation  among  both  program  and  control  group 
members;  in  some  other  cases,  these  differences  were  small  for  another  reason:  higher-than- 
expected  rates  of  service  receipt  or  participation  by  the  control  group  members  that  were  not 
exceeded  by  the  rates  of  the  program  group  members.  In  cases  where  there  were  small  “treat- 
ment” differences,  the  probability  of  achieving  positive  effects  on  retention  and  advancement 
was  lowered. 

All  the  above  factors  — the  ways  in  which  the  ERA  models  deviated  from  their  designs 
in  their  actual  implementation,  the  levels  of  participation  by  program  group  members  in  various 
types  of  services  and  activities,  and  the  contrasts  between  the  program  and  control  group 
members’  participation  in  the  different  services  and  activities  — need  to  be  taken  into  account 
when  interpreting  the  impact  findings  presented  in  this  report.  In  the  conclusions  of  the  presen- 
tations of  findings  for  each  ERA  model  in  this  report,  each  of  these  factors  is  discussed,  as  a 
way  of  helping  readers  interpret  the  economic  impacts  for  that  model.  In  addition,  the  last 
chapter  of  the  report  uses  these  factors  to  broadly  assess  the  impacts  for  groupings  of  ERA 
models. 


The  Structure  of  This  Report 

The  structure  of  the  following  chapters  is  designed  to  allow  readers  to  take  several 
pathways  through  the  report,  given  that  this  report  serves  as  the  foundation  for  a number  of 
analytical  papers,  syntheses,  and  practice  briefs  that  will  be  forthcoming  from  the  ERA  project. 
The  report’s  structure  is  outlined  below. 

Chapter  2 presents  information  about  the  ERA  project’s  research  samples  and  data 
sources.  It  also  describes  the  key  implementation  and  economic  outcome  measures  that  are  used 
throughout  the  report. 

Following  Chapter  2,  results  are  presented  in  three  separate  chapters  that  group  the 
ERA  models  according  to  the  status  of  their  target  populations  as  of  study  entry.  Each  chapter 
first  discusses  a model’s  history,  target  population,  and  significance;  then  it  presents  the  model’s 
implementation  features  and  economic  impacts. 

The  groupings  of  ERA  models,  and  the  chapters  covering  them,  are  as  follows: 

• Models  serving  unemployed  TANF  recipients  (Chapter  3).  Three  models 
first  provided  services  when  individuals  were  not  employed  and  were  receiv- 
ing TANF:  the  Texas  model  (operated  in  three  sites),  the  Los  Angeles  En- 
hanced Job  Club  (EJC)  model,  and  the  Salem  model. 
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• Models  serving  employed  TANF  recipients  (Chapter  4).  Four  models  first 
provided  services  when  individuals  were  employed  and  were  receiving 
TANF:  the  Chicago,  Los  Angeles  Reach  for  Success  (RFS),  Riverside  Train- 
ing Focused,  and  Riverside  Work  Plus  models. 

• Models  serving  employed  non-TANF  recipients  (Chapter  5).  Five  models 
first  provided  services  when  individuals  were  employed  and  not  receiving 
TANF:  the  Cleveland,  Eugene,  Medford,  Riverside  Post- Assistance  Self- 
Sufficiency  (PASS),  and  South  Carolina  models.35 

Because  the  economic  effects  that  are  measured  in  the  ERA  project  reflect  what  actual- 
ly happened  to  individuals  when  they  were  eligible  for  the  ERA  models,  the  focus  of  the 
implementation  discussions  in  the  above  three  chapters  is  on  the  services  that  were  actually 
delivered  as  sites  implemented  these  models  — not  on  what  was  designed  to  occur.  Implemen- 
tation issues  were  common,  although  they  varied  in  their  severity. 

Chapter  6,  the  final  chapter  of  the  report,  presents  a summary  of  the  ERA  effectiveness 
results  for  each  of  the  three  groupings  of  models  and  provides  some  concluding  thoughts. 

Five  appendixes  support  the  results  presented  in  this  report.  Appendixes  A,  B,  and  C 
contain  tables  of  results  that  are  drawn  on  heavily  in  the  main  results  chapters.  Appendix  D 
includes  tables  that  show  secondary  measures  from  the  42 -month  survey  and  also  presents  the 
42 -month  survey  response  analysis.  Appendix  E contains  citations  for  the  ERA  interim  reports 
that  discuss  the  program  models  and  sites  included  in  this  report.  Box  1.2  explains  how  to 
interpret  the  impact  tables  presented  in  the  ERA  evaluation. 


35While  included  in  this  grouping,  the  South  Carolina  model  served  both  employed  and  nonemployed 
TANF  leavers.  The  model  sought  to  find  and  provide  services  to  individuals  who  had  left  TANF  up  to  three 
years  earlier.  When  contacted  by  program  staff,  some  of  these  individuals  were  not  employed. 
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Box  1.2 


How  to  Read  the  Impact  Tables  in  the  ERA  Evaluation 

Most  tables  in  this  report  use  a similar  format,  illustrated  below.  The  data  show  a series  of 
participation  outcomes  for  the  ERA  group  and  the  control  group.  For  example,  the  table 
shows  that  about  60  (60.2)  percent  of  the  ERA  group  and  about  48  (47.7)  percent  of  the 
control  group  “ever  had  contact  with  staff/employment  program.” 

Because  individuals  were  assigned  randomly  either  to  the  ERA  group  or  to  the  control  group, 
the  effects  of  the  program  can  be  estimated  by  the  difference  in  outcomes  between  the  two 
groups.  The  “Difference  (Impact)”  column  in  the  table  shows  the  differences  between  the  two 
research  groups’  participation  rates  — that  is,  the  program’s  impacts  on  participation.  For 
example,  the  impact  on  “ever  had  contact  with  staff/employment  program”  can  be  calculated 
by  subtracting  47.7  percent  from  60.2  percent,  yielding  12.5  percentage  points. 

Differences  marked  with  asterisks  are  “statistically  significant,”  meaning  that  it  is  quite 
unlikely  that  the  differences  arose  by  chance.  The  number  of  asterisks  indicates  the  level  of 
statistical  significance  of  the  impact  (the  lower  the  level,  the  less  likely  that  the  impact  is  due 
to  chance).  One  asterisk  corresponds  to  the  10  percent  level;  two  asterisks,  the  5 percent 
level;  and  three  asterisks,  the  1 percent  level.  The  p-values  show  the  exact  levels  of  signific- 
ance. For  example,  as  shown  below,  the  ERA  group  had  a statistically  significant  impact  of 
12.5  percentage  points  at  the  5 percent  level  on  the  measure  “ever  had  contact  with 
staff/employment  program.” 


Impacts  on  Participation  Outcomes 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

60.2 

47.7 

12.5  ** 

0.041 

Average  number  of  contacts  with  staff 

6.6 

6.4 

0.3 

0.889 

Talked  with  staff  in  past  4 weeks  (%) 

27.2 

16.5 

10.7  ** 

0.036 

Chapter  2 

Sample  Sizes,  Data  Sources,  and  Measures 


This  chapter  provides  background  information  on  the  research  samples,  data  sources, 
and  outcome  measures  that  are  used  in  the  Employment  Retention  and  Advancement  (ERA) 
project  and  are  discussed  in  the  following  three  chapters.  Chapter  1 demonstrates  the  diversity 
and  innovation  of  the  ERA  programs,  in  temis  of  their  goals,  strategies,  and  target  populations. 
In  order  to  capture  and  systematically  analyze  this  diversity,  a standardized  set  of  implementa- 
tion and  economic  outcome  measures  are  used  in  this  report.  The  implementation  measures 
convey  what  was  actually  tested  in  ERA,  and  they  facilitate  systematic  descriptions  of  how  the 
ERA  programs  rolled  out  and  the  extent  to  which  services  were  used.  The  economic  measures 
assess  improvements  in  retention  and  advancement  outcomes.  The  chapter  begins  with  an 
overview  of  sample  sizes  and  data  sources  and  then  discusses  implementation  and  economic 
outcome  measures. 


Sample  Sizes  and  Data  Sources 

Sample  Sizes 

Figure  2.1  shows  the  random  assignment  dates,  data  follow-up  periods,  and  sample  siz- 
es for  each  of  the  ERA  tests  discussed  in  this  report.  Large  sample  sizes  enable  reliable  esti- 
mates of  the  program  impacts.  As  noted  in  Chapter  1,  the  ERA  evaluation  followed  over  45,000 
individuals.  The  sample  followed  for  this  report  is  smaller  because  this  report  does  not  include 
results  for  three  models  that  served  harder-to-employ  individuals  and  does  not  include  outcomes 
for  two-parent  families.  Results  for  these  models  and  sample  members  will  be  shared  in  future 
documents. 

Approximately  27,000  individuals  are  followed  for  this  report,  and  sample  members  are 
spread  fairly  evenly  over  tests  in  the  three  target  groups  relating  to  employment  or  nonemploy- 
ment and  receipt  of  Temporary  Assistance  for  Needy  Families  (TANF):  unemployed  TANF 
recipients  = 8,018;  employed  TANF  recipients  = 10,457;  and  employed  non-TANF  recipients 
= 8,586. 


Considering  the  tests  that  are  covered  in  this  report,  the  earliest  began  random  assign- 
ment in  October  2000  (the  Corpus  Christi  and  Fort  Worth  sites  of  the  Texas  ERA  model),  and 
the  latest  concluded  random  assignment  in  September  2004  (the  Los  Angeles  Enhanced  Job 
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The  Employment  Retention  and  Advancement  Project 

Figure  2.1 

Timelines  and  Sample  Sizes,  by  Site 


SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTE:  The  42-month  survey  was  administered  only  in  Chicago,  Los  Angeles  RES,  and  Riverside  PASS. 


Random  assignment  period 
Follow-up  period 


Club  [EJC]  model).  Across  the  ERA  tests,  sample  sizes  range  from  697  (in  the  Cleveland  ERA 
test)  to  5,700  (in  the  Los  Angeles  Reach  for  Success  [RFS]  ERA  test).1  For  each  of  the  tests, 
analyses  were  conducted  to  confinn  that  ERA  group  members  were  statistically  indistinguish- 
able from  control  group  members  at  the  time  of  random  assigmnent,  that  is,  that  the  random 
assignment  process  “worked.”  Analyses  confirmed  this  in  every  case. 

Data  Sources 

The  measures  discussed  in  this  chapter  and  used  throughout  this  report  were  created 
from  the  following  data  sources. 

Baseline  Data 

Baseline  data,  collected  at  the  time  of  random  assignment,  include  information  on  de- 
mographic characteristics,  prior  employment,  and  prior  welfare  receipt.  This  infonnation  is  used 
to  describe  the  study  samples,  to  create  subgroups,  and  to  refine  statistical  estimates. 

Field  Research 

Field  research  is  the  main  data  source  for  the  analysis  of  program  features  and  imple- 
mentation. MDRC  staff  interviewed  case  managers,  service  providers,  and  program  administra- 
tors to  leam  about  the  goals,  structure,  and  operations  of  the  ERA  programs  at  several  points 
over  the  course  of  the  study  period.  MDRC  researchers  collected  infomiation  on  a range  of 
topics,  including  the  marketing  and  outreach  approaches  used  to  recruit  prospective  participants, 
the  types  of  program  services  and  supportive  services  provided,  management  and  staffing 
structure,  relationships  between  organizations  involved  in  program  operations,  enforcement  of 
any  participation  requirements,  and  implementation  issues.  As  part  of  this  work,  MDRC  staff 
also  reviewed  a number  of  sample  members’  case  files  and  observed  some  program  operations. 
In  addition,  some  focus  groups  were  conducted.  Finally,  a study  of  program  staffs  use  of  time 
was  completed. 

Administrative  Records 

Effects  on  employment  and  earnings  were  computed  using  unemployment  insurance 
(UI)  wage  records  data,  and  effects  on  public  assistance  were  computed  using  TANF  and  food 
stamp  administrative  records.  All  data  sources  cover  only  activity  within  the  states  in  which  the 
evaluations  were  conducted.  As  shown  in  Figure  2.1,  follow-up  data  are  available  through  2007 
for  most  models.  Four  years  of  post-random  assigmnent  UI  wage  data  are  available  for  all 


'The  sample  sizes  in  this  report  for  South  Carolina  differ  from  those  stated  in  the  interim  report  because,  as 
noted,  two-parent  families  are  excluded  from  the  analyses  in  this  report. 
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sample  members  in  half  the  models;  in  the  other  half,  three  years  are  available.  UI  records  cover 
only  employment  that  is  reported  to  the  unemployment  insurance  system.  It  is  estimated  that  UI 
data  cover  approximately  90  percent  of  all  jobs.2  UI  coverage  varies  among  states  but  generally 
excludes  most  federal,  railroad,  and  agriculture  employees  as  well  as  family  workers,  domestics, 
and  independent  contractors.  In  addition,  all  unreported  employment  is  missed.  Four  years  of 
post-random  assignment  food  stamp  and  TANF  data  are  available  for  five  models,  and  three 
years  of  data  are  available  for  six  models.  (No  welfare  data  were  collected  for  the  Cleveland  test 
because  its  sample  had  no  direct  association  with  TANF  as  of  the  time  of  study  entry.) 

Labor  Market  Information 

Data  on  local  labor  markets  were  collected  from  the  U.S.  Department  of  Labor,  Bureau 
of  Labor  Statistics,  in  order  to  profile  labor  market  conditions  at  the  times  and  places  that  the 
ERA  programs  operated.  Data  for  these  calculations  were  drawn  from  the  Local  Area  Unem- 
ployment Statistics  program  and  the  Quarterly  Census  of  Employment  and  Wages. 

12-Month  Survey  Data 

Information  about  sample  members’  experiences  with  program  operations  and  services 
as  well  as  their  employment  outcomes  was  gathered  through  a survey,  which  was  administered 
to  a subset  of  ERA  and  control  group  members  approximately  12  months  after  their  random 
assignment.  The  numbers  of  respondents  to  this  survey  across  the  model  tests  range  from  188 
sample  members  to  848  sample  members  (Figure  2.1).  Response  rates  varied  from  72  percent  in 
Salem,  Oregon,  to  82  percent  in  Corpus  Christi. 

For  most  sites,  the  response  bias  analysis  of  the  12-month  survey  indicates  some  differ- 
ences in  background  characteristics  of  varying  levels  between  the  survey  sample  and  the 
research  sample,  between  the  program  and  the  control  group  members  in  the  survey  sample, 
and/or  between  the  survey  respondents  and  the  nonrespondents.  Except  for  the  situation  in  the 
survey  for  the  Post-Assistance  Self-Sufficiency  (PASS)  program  in  Riverside,  California, 
however,  none  of  these  differences  is  of  a nature  that  seriously  affects  the  generalization  of  the 
survey  results  pertaining  to  participants’  experiences  with  program  operations  and  services.  In 
the  case  of  Riverside  PASS,  the  survey  was  drawn  from  a cohort  that  experienced  no  large 
economic  impacts,  while  the  full  sample  experienced  increases  in  a wide  range  of  measures.  In 
addition,  the  survey  sample  for  Riverside  PASS  is  small  (224)  and  not  representative  of  the  full 
research  sample. 


2Komfeld  and  Bloom  (1999). 
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For  findings  related  to  economic  outcomes,  the  administrative  records  are  given  more 
weight  than  the  12-month  survey,  for  several  reasons.  First,  the  survey  findings  are  usually 
based  on  a relatively  small  sample  taken  from  select  cohorts  of  the  research  sample,  compared 
with  the  findings  based  on  the  administrative  records,  which  cover  the  entire  research  sample. 
Second,  several  sites  had  differences  between  survey  respondents  and  nonrespondents  related  to 
previous  employment,  earnings,  and/or  public  assistance  receipt.  Finally,  the  survey  data  are 
subject  to  recall  error  among  respondents.  Therefore,  greater  weight  is  placed  on  the  findings 
based  on  administrative  records  in  this  evaluation,  particularly  in  cases  where  the  survey  results 
are  different. 

42-Month  Survey  Data 

A second  survey  was  administered  to  sample  members  in  the  Chicago,  Los  Angeles 
RFS,  and  Riverside  PASS  ERA  tests  42  months  after  enrollment  in  the  program.  The  adminis- 
tration of  the  42 -month  survey  was  constrained  to  a small  number  of  sites  for  resource  reasons; 
these  three  sites  were  chosen  because  of  their  promising  results  as  of  the  time  of  survey  site 
selection.3  Respondent  sample  sizes  range  from  nearly  900  in  the  Riverside  PASS  test  to 
approximately  1 ,000  in  the  other  two  tests,  and  no  serious  response  bias  problems  were  detected 
(Figure  2.1).  Response  rates  varied  from  68  percent  in  Riverside  PASS  to  78  percent  in  Chica- 
go. The  42 -month  survey  covered  employment  retention  and  advancement  in  greater  depth  than 
the  12-month  survey,  ft  also  included  questions  pertaining  to  other  areas,  such  as  health  and 
family  outcomes.  Impacts  on  these  outcomes  are  shown  in  Appendix  D. 


Measures 

Implementation 

Implementation  research  supports  impact  analyses  by  describing  the  nature  of  the  pro- 
grams being  tested  and  providing  possible  explanations  for  the  patterns  of  impacts.  It  also 
provides  information  that  serves  as  the  foundation  for  the  next  generation  of  models  and 
programs.  The  examination  of  the  implementation  of  the  ERA  programs  in  this  report  focuses 
on  the  resources  and  organizational  capacity  of  providers,  the  operating  procedures  and  systems 
in  the  delivery  of  services,  and  the  uptake  of  services  by  the  target  population.  A series  of 


3When  decisions  about  survey  selection  were  made,  economic  impacts  in  the  Texas  ERA  site  appeared  to 
be  fading.  Later  in  the  follow-up  period,  however,  the  impacts  strengthened  again,  particularly  in  the  Corpus 
Christi  site. 
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implementation  measures  are  used  to  systematically  assess  the  programs  along  two  dimensions: 
program  features  and  client  experiences.4 

Program  Features 

The  specific  program  features  examined  in  the  report  are  described  in  Box  2.1.  The  first 
two  features  — program  resources  and  organization,  and  staffing  — provide  information  on  the 
logistics  of  administering  ERA  programs.  The  others  capture  the  breadth  of  strategies  both 
within  and  across  ERA  programs.  Systematic  field  research  served  as  the  primary  data  source 
for  assessing  ERA  program  features. 

Throughout  the  following  chapters,  the  details  of  features  for  each  ERA  program  are 
presented.  In  addition,  differences  between  the  ERA  programs  and  the  counterfactual  situations 
experienced  by  control  group  members  are  assessed  for  each  program  feature.  (The  outcomes 
for  people  who  were  randomly  assigned  to  the  control  groups  represent  the  counterfactual 
because  the  control  groups  could  not  participate  in  ERA  services;  their  outcomes  thus  represent 
what  would  have  happened  in  the  absence  of  the  ERA  programs.) 

Client  Experiences 

The  measures  of  client  experience  included  in  the  report  were  chosen  to  capture  the  up- 
take of  services  of  both  ERA  and  control  group  members  and  are  tied  to  the  program  features 
discussed  above.  ERA  sought  to  provide  help  that  went  well  beyond  the  traditional  services 
provided  by  welfare- to- work  programs.  These  new  service  strategies  were  rarely  captured  by 
existing  management  information  systems,  and  it  was  possible  that  control  group  members 
could  have  received  them  from  entities  or  initiatives  outside  ERA.  Therefore,  surveys  (particu- 
larly the  12-month  survey)  are  the  key  data  source  for  measuring  participation  in  ERA  services 
and  for  comparing  participation  levels  of  the  program  and  control  groups. 

Fully  capturing  ERA  service  levels  through  surveys,  however,  has  some  drawbacks. 
Some  of  the  provided  ERA  services  could  have  been  perceived  by  participants  as  relatively 
insignificant  (for  example,  a short  telephone  conversation  to  check  in  about  problems  on  the 


4ln  this  report,  the  term  staff-client  engagement  (Box  2.1)  is  used  to  capture  the  staff  activities  traditionally 
associated  with  case  management.  Using  the  term  “case  management”  often  implies  uniformity  in  the 
individualized  services  provided  by  program  staff,  even  though  there  may  be  important  differences  across 
models  and  sites.  It  also  is  a term  that  many  human  service  practitioners  use  to  describe  a more  narrow  scope  of 
activities  than  those  conducted  by  ERA  staff.  Thus,  in  this  report,  the  way  ERA  staff  worked  with  clients  on  an 
individualized  basis  is  treated  as  one  programmatic  feature  within  the  broader  range  of  services  that  were 
provided.  The  specific  details  regarding  the  way  staff  worked  with  clients  — what  was  discussed  in  interac- 
tions, what  approach  was  used,  the  intensity  of  the  interactions,  and  staffing  configurations  — is  described,  so 
that  different  enhancements  across  sites  can  be  articulated. 
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Box  2.1 


Program  Features  Assessed  for  the  ERA  Programs 

Program  resources  and  organization.  The  sources  of  financial  resources  and  the  setup  of 
institutional  structures,  including  the  agencies  or  partnerships  providing  program  adminis- 
tration and  the  range  of  service  providers  enlisted  for  the  oversight  and  delivery  of  ERA 
programs. 

Staffing.  The  human  capital  resources  drawn  upon  for  the  delivery  of  ERA  programs, 
including  staffing  levels;  staffing  arrangements  (for  example,  team-based  staffing);  the 
duties  associated  with  various  staff  positions;  the  level  and  nature  of  staff  experience, 
skills,  and  training;  performance  standards  and  incentives;  and  staff  practices  in  terms  of 
meeting  locations  and  work  schedules. 

Staff-client  engagement.  The  nature  of  staff-client  interactions  within  the  ERA  programs, 
from  initial  outreach  through  ongoing  participation,  including  the  rules  of  any  participation 
requirements;  the  means,  tools,  and  intensity  of  marketing  and  outreach  efforts;  how 
clients  were  directed  through  program  services  and  activities  and  the  use  of  formal  or 
informal  assessments;  and  the  level  and  ways  of  monitoring  of  client  progress  and  con- 
ducting follow-up. 

Initial  job  preparation  and  placement  services.  The  nature,  content,  and  emphases  of  ERA 
services  for  moving  unemployed  clients  into  jobs. 

Retention  services.  The  nature  and  content  of  ERA  services  and  supports  for  stabilizing  and 
maintaining  employment  among  the  newly  employed  or  those  who  recently  changed  jobs, 
including  connections  to  work  supports  and  social  service  providers,  one-time  payments  to 
cover  financial  needs,  financial  work  incentives,  and  counseling  on  job-related  issues.  Included 
here  are  also  the  reemployment  activities  directed  to  individuals  who  became  unemployed 
(similar  to  services  offered  for  initial  job  placement). 

Advancement  services.  The  content  and  nature  of  ERA  activities  promoting  advancement, 
including  career  counseling,  counseling  regarding  how  to  seek  a promotion  or  increase  in 
wages  or  work  hours  at  a current  job,  job  search  and  job  development  activities  to  identify 
“better”  or  higher-paying  jobs,  and  referrals  to  and  participation  incentives  for  education  and 
training  programs  to  build  skills  and  credentials. 

Employer  linkages.  The  formal  and  informal  relationships  between  ERA  program  staff  and 
local  employers,  including  the  collection  of  information  about  local  employers’  labor  needs, 
coaching  clients  in  the  job  application  process  for  specific  employers,  job  development  efforts, 
and  discussing  clients’  performance  and  possible  career  ladders  with  employers’  staff. 
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job),  even  if  they  influenced  participants’  behavior.  It  is  likely  that  some  clients  did  not  recall, 
and  thus  did  not  report,  some  of  these  interactions  with  ERA  staff.5 

Measures  of  client  experiences  are  briefly  summarized  in  Box  2.2. 6 Specifically,  client 
experiences  include  contact  with  staff  from  any  type  of  welfare  or  employment  program  and  the 
type  of  help  or  services  received  from  such  staff.  Program-control  comparisons  of  these  meas- 
ures indicate  how  client  experiences  differed  between  the  ERA  and  control  groups,  that  is,  the 
extent  to  which  there  were  impacts  on  these  client  experience  outcomes. 

Economic  Outcomes 

The  economic  impacts  analysis  in  this  report  focuses  on  three  categories  of  outcomes: 
(1)  employment  retention,  (2)  earnings  and  advancement,  and  (3)  public  assistance  and  income. 
As  discussed  in  Chapter  1 , research  on  employment  retention  and  advancement  is  still  relatively 
new  for  this  policy  area.  In  addition,  employment  retention  and  advancement  are  complex 
behaviors  requiring  multiple  indicators  in  order  to  determine  whether  a program  has  affected 
them.  Therefore,  new  measures  were  created  in  order  to  capture  whether  the  ERA  programs 
affected  retention  and  advancement.  These  measures  are  presented  in  Box  2.3  and  are  discussed 
below. 


Employment  Retention 

“Employment  retention”  refers  to  the  extent  of  an  individual’s  labor  force  participation. 
ERA  programs  were  deemed  as  impacting  employment  retention  if  the  amount  of  employment 
observed  for  the  ERA  group  exceeded  the  amount  of  employment  observed  for  the  control 
group.  The  amount  of  employment  can  be  assessed  in  a variety  of  ways.  First,  a program  could 
increase  the  amount  of  time  that  ERA  group  members  were  employed  simply  by  increasing  the 
extent  to  which  ERA  group  members  were  ever  employed.  Differences  between  the  ERA  and 
the  control  groups  in  the  cumulative  amount  of  employment  over  the  follow-up  period  can  also 
reflect  an  impact  on  the  frequency  of  employment.  Finally,  increasing  the  stability  of  ERA 
group  members’  employment  also  increases  the  amount  of  employment,  by  reducing  the 
occurrence  of  unemployment.  Programs  could  possibly  affect  employment  along  some  of  these 


5There  is  evidence  that  this  was  an  issue,  at  least  in  some  sites:  data  available  from  the  management  in- 
formation systems  in  a few  of  the  ERA  sites  indicate  higher  levels  of  participation  in  the  ERA  services  by 
program  group  members  than  is  indicated  by  survey-based  estimates  of  participation. 

6Many  other  measures  of  client  experiences  are  analyzed  in  the  ERA  site-specific  interim  reports.  For  ex- 
act citations,  see  Appendix  E.  The  reports  can  be  found  on  the  Web  at  the  sites  for  the  Administration  for 
Children  and  Families  (http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/index.html) 
and  MDRC  (http://www.mdrc.org/project_publications_14_9.html). 
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Box  2.2 


Client  Experiences  Assessed  for  the  ERA  Programs 

Extent  of  engagement.  The  extent  of  staff  engagement  with  sample  members,  including  the 
frequency,  and  ongoing  nature  of  contacts. 

Received  help  with  retention  services.  The  extent  and  nature  of  assistance  related  to 
employment  retention  that  was  received  by  sample  members,  including  participation  in  job 
search  activities  and  help  dealing  with  problems  on  the  job. 

Received  help  with  employment  advancement  services.  The  extent  and  nature  of  assis- 
tance with  employment  advancement  that  was  received  by  clients,  including  help  with  career 
assessment  and  participation  in  any  education  or  training  activities. 


dimensions  and  not  others,  and  so  attention  to  the  pattern  of  impacts  along  all  these  dimensions 
provides  an  understanding  as  to  how  the  ERA  programs  affected  employment  retention. 

The  primary  focus  for  the  analysis  of  impacts  on  employment  retention  is  on  overall 
employment,  regardless  of  whether  the  employment  was  with  a single  employer  or  multiple 
employers.  Impacts  on  job-level  retention  are  shown  in  Appendixes  A,  B,  and  C. 

Earnings  and  Advancement 

In  this  report,  “advancement”  is  defined  primarily  as  increases  in  earnings  that  are  not 
fully  explained  by  increases  in  quarters  of  employment  and,  secondarily,  as  improvements  in 
fringe  benefits  received,  working  conditions,  or  opportunities  for  promotions.  In  the  three  42- 
month  survey  sites,  it  is  possible  to  look  at  these  secondary  advancement  measures  in  more 
depth  and  over  a time  period  within  which  improvements  in  advancement  outcomes  could  be 
reasonably  expected. 

Increases  in  earnings  provide  a clear  and  direct  measure  of  economic  mobility.  Howev- 
er, earnings  can  increase  for  reasons  other  than  advancement.  This  is  because  earnings  levels  are 
directly  impacted  by  employment  rates.  All  else  being  equal,  increases  in  any  of  the  employ- 
ment retention  measures  discussed  in  the  previous  section  will  increase  earnings.  This  is 
because  average  earnings  include  zeros  for  people  who  were  not  working  during  the  follow-up 
period.  Consequently,  increases  in  the  percentage  of  the  follow-up  period  employed  will 
directly  translate  into  earnings  increases.  In  the  impact  analysis,  an  effort  is  made  to  “tease  out” 
the  increases  in  earnings  that  are  due  to  working  more  quarters  from  increases  in  earnings  that 
are  due  to  other  factors  (such  as  working  for  more  hours,  or  at  higher  wages,  or  for  more 
weeks).  One  indicator  of  advancement  is  seen  when  increases  in  earnings  are  larger,  in  percent- 
age terms,  than  increases  in  employment  retention. 
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Box  2.3 


Employment  Retention  and  Advancement  Measures 
Assessed  for  the  ERA  Programs 

Employment  Measures 

Ever  employed  (%).  This  is  the  percentage  of  sample  members  that  ever  worked  in  a UI- 
covered  job  during  the  follow-up  period. 

Average  quarterly  employment  (%>).  This  cumulative  measure  of  employment  examines, 
overall,  how  much  one  worked  during  the  follow-up  period.  The  average  quarterly  employ- 
ment measure  does  not,  however,  capture  the  extent  to  which  individuals  held  onto  employ- 
ment in  consecutive  quarters.  It  measures  the  frequency  of  employment. 

Had  employment  spell  of  at  least  four  quarters  (%>).  This  is  a measure  of  continuous  employ- 
ment retention  (in  consecutive  quarters).  It  measures  the  stability  of  employment. 

Length  of  longest  employment  spell  (in  quarters).  This  is  another  measure  of  the  stability  of 
employment.  Inversely,  the  length  of  longest  unemployment  spell  measures  how  long  individ- 
uals stayed  out  of  the  labor  market  and  provides  a measure  of  how  quickly  the  ERA  programs 
reemployed  those  who  lost  their  jobs. 

Earnings  and  Advancement  Measures 

Average  annual  earnings  ($).  This  measure  can  be  affected  by  the  number  of  quarters,  weeks, 
or  hours  worked,  and  wage  rates.  Increases  in  earnings  suggest  the  possibility  of  advancement, 
but  other  measures  need  to  be  assessed  in  order  to  complete  the  determination. 

Quarters  with  earnings  of  $3,500  or  more  (%).  This  measure  indicates  the  percentage  of 
quarters  in  which  sample  members  earned  over  $3,500.  This  level  of  earnings  corresponds  to 
working  full  time,  at  a wage  of  $7.50  per  hour,  for  all  of  the  weeks  in  a quarter.  For  many 
sample  members,  earning  at  this  level  would  represent  advancement  in  the  labor  market. 

Comparison  of  quarter  with  highest  earnings  in  Year  1 and  quarter  with  highest  earnings  in 
Year  3 or  Year  4 that  increased  by  $250  or  more.  This  measure  picks  up  long-term  advance- 
ment. If  ERA  had  immediate  effects  on  advancement  in  Y ear  1 (as  appeared  to  occur  in  a few 
sites)  and  these  effects  do  not  grow  over  time,  they  will  not  show  up  in  this  measure. 

Other  advancement  measures.  Advancement  is  primarily  defined  in  this  report  as  increases  in 
earnings  that  are  not  fully  explained  by  increases  in  employment.  In  addition,  improvements 
in  fringe  benefits  received,  working  conditions,  or  opportunities  for  promotions  are  examined. 
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As  noted  above,  most  of  the  economic  impact  analyses  in  this  report  use  unemployment 
insurance  records.  Because  UI  data  do  not  typically  contain  information  on  wages  or  on  hours 
or  weeks  worked,  these  data  do  not  directly  indicate  why  earnings  increases  might  be  larger  than 
employment  increases.  The  UI  data  cannot  help  to  quantify  the  relative  contribution  of  increases 
in  wages,  hours,  or  weeks  worked.  In  the  sites  with  42-month  survey  data  available,  it  is 
possible  to  make  more  precise  statements  as  to  which  factors  are  driving  the  results. 

Public  Assistance  and  Income 

Since  most  of  the  ERA  sample  members  were  current  or  former  welfare  recipients  at 
the  time  of  random  assigmnent,  the  report  also  examines  how  these  employment-focused 
programs  affected  the  use  of  public  benefits.  In  addition,  earnings  increases  may  be  completely 
offset  or  supplemented  by  changes  in  public  assistance,  so  effects  on  total  income  are  analyzed. 
Most  of  the  discussion  of  income  in  this  report  focuses  on  the  sum  of  TANF  and  food  stamp 
payments  and  earnings.  A more  complete  measure  of  income  — which  includes  income  from 
jobs  not  covered  by  the  UI  system  and  income  from  other  household  members,  child  support, 
Supplemental  Security  Income  [SSI],  and  other  sources  — is  available  from  the  12-month 
survey  and  the  42 -month  survey.7  Impacts  using  this  measure  of  income  are  presented  in 
Appendixes  A,  B,  and  C. 

Secondary  Outcomes 

Other  economic  and  noneconomic  outcomes  were  tracked  as  part  of  the  12-  and  42- 
month  surveys.  These  include  measures  of  child  outcomes,  adult  health,  household  composi- 
tion, and  other  sources  of  income.  Impacts  on  several  of  these  measures  are  shown  in  Appendix 
D.  These  measures,  while  important,  do  not  reflect  the  direct  “primary  outcomes”  that  ERA 
sought  to  affect.  (Those  are  employment  retention  and  advancement.)  Therefore,  impacts  on 
these  measures  should  be  viewed  with  some  caution,  particularly  in  tests  where  ERA  did  not 
affect  the  primary  outcomes  that,  in  turn,  would  be  expected  to  influence  them.  Regarding  the 
secondary  outcomes  in  general,  it  is  important  to  be  aware  of  the  possibility  that  some  compari- 
sons will  be  statistically  significant  by  chance,  given  the  many  secondary  outcomes. 


7In  addition,  a fuller  measure  that  includes  all  measurable  sources  of  income  is  presented  in  a companion 
report  on  the  benefits  and  costs  of  selected  ERA  programs  (Redcross,  Deitch,  and  Farrell,  2010). 
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Chapter  3 


Findings  for  ERA  Programs  Serving 
Unemployed  TANF  Recipients 


Introduction 

The  Employment  Retention  and  Advancement  (ERA)  programs  that  are  discussed  in 
this  chapter  include  those  that  targeted  individuals  who  were  not  working  and  were  applicants 
or  recipients  of  Temporary  Assistance  for  Needy  Families  (TANF)  when  they  enrolled  in  the 
programs.  This  category  includes  three  programs:  the  Texas  ERA  program,  which  operated  in 
Corpus  Christi,  Fort  Worth,  and  Houston;  the  Enhanced  Job  Club  (EJC)  model  in  Los  Angeles; 
and  the  Salem  (Oregon)  ERA  program. 

Among  the  programs  described  in  this  chapter,  only  the  Texas  program  as  operated  in 
Corpus  Christi  and  Fort  Worth  produced  improvements  in  employment  retention  and  advance- 
ment. The  ERA  program  in  Corpus  Christi  increased  employment  retention  and  led  to  ad- 
vancement gains,  compared  with  the  levels  seen  in  the  control  group.  It  also  reduced  the  amount 
of  food  stamp  assistance  that  clients  received,  and  it  increased  their  average  annual  income  in 
the  cumulative  four-year  follow-up  period.  The  ERA  program  in  Fort  Worth  produced  a similar 
pattern  of  results,  except  that  its  effects  were  somewhat  less  persistent  over  time. 


Overview  of  Program  Designs  in  This  Group 

All  the  programs  discussed  in  this  chapter  targeted  unemployed  TANF  applicants  and 
recipients  and  shared  several  goals,  including  improving  the  job  placement,  job  retention,  and 
employment  advancement  outcomes  of  program  group  members.  Because  the  programs 
targeted  cash  assistance  applicants  and  recipients  who  were  not  working  when  they  enrolled  in 
the  program,  they  had  a common  strategy  of  providing  preemployment  services  that  individuals 
were  required  to  participate  in  as  a condition  of  receiving  cash  assistance.  However,  there  were 
differences  in  each  program’s  postemployment  services. 

Table  3.1  compares  a selection  of  actual  (as  opposed  to  planned)  program  features  for 
the  ERA  programs  aimed  at  this  target  group.  As  the  table  shows,  within  the  programs,  some 
features  were  particularly  prominent  and,  therefore,  represent  the  focus  of  the  discussion  in  this 
report.  Appendix  A briefly  describes  all  the  program  features  (Appendix  Boxes  A.l,  A.2,  and 
A.3)  and  provides  details  on  program  administration  (such  as  operation  dates  and  funding).  This 
chapter  discusses  the  key  program  services  as  they  actually  were  implemented  and  how  these 
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The  Employment  Retention  and  Advancement  Project 
Table  3.1 

Comparison  of  Program  Features  Across  ERA  Programs  Targeting  Individuals 
Not  Employed  and  Receiving  TANF 


Program  Feature 


0° 


T 


So 


V 


Staffing 

Staff  positions  - team-based  (vs.  solo) 

>/ 

Staff  performance  management  (standards,  incentives) 

Flexible  staff  hours  and/or  locations 

Caseloads  - low 

Staff/client  engagement 

Participation  requirements 

TANF  participation  requirements 

ERA  participation  requirements  beyond  TANF  ones 

Initial  outreach  - intensive 

Initial  outreach  - financial  participation  incentives 

Program  flow  - client  assessment 
and  individualized  service  package 

Ongoing  - develop/maintain  an  employment  plan 

Initial  job  preparation  and  placement  services 

>/ 

>/ 

Retention  services 

Reemployment  services 

Financial  work  incentives 

Emergency  financial  assistance 

Connections  to  other  services  (work  supports,  social  services) 

>/ 

>/ 

>/ 

Staff  counseling  on  job-related  issues 

Advancement  services 

Supported  advancement  through  job  change 

Education  and  training  referrals  and/or  incentives 

Staff  counseling  on  job-related  issues 

>/ 

Employer  linkages 

>/ 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 

NOTES:  The  program  features  are  defined  in  Chapter  2. 

Check  marks  indicate  that  the  feature  is  present  in  the  ERA  program. 

The  models  and  their  implementation  often  evolved  over  the  study  period.  This  table  presents  the  features 
experienced  by  the  majority  of  the  study  participants  for  the  greatest  extent  of  time. 
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differed  from  the  services  received  by  the  control  group.  A broader  discussion  of  both  planned 
and  actual  implementation  is  provided  in  the  ERA  site-specific  interim  reports. 1 

One-on-one  staff-client  relationships  were  central  to  all  the  programs  except  for  the  Los 
Angeles  EJC  model.  Unlike  the  other  programs,  Los  Angeles  EJC  used  job  clubs  to  deliver 
services  to  groups  of  ERA  clients.  While  each  program  was  designed  to  deliver  postemploy- 
ment services,  the  staffing  approach  and  intensity  varied.  For  example,  the  Los  Angeles  EJC 
ERA  model  did  not  include  any  postemployment  services  by  design,  but  it  tried  to  achieve 
retention  and  advancement  goals  by  placing  participants  in  jobs  in  their  field  of  interest.  In 
contrast,  the  Salem  and  the  Texas  programs  invested  staff  resources  in  the  delivery  of  postem- 
ployment services  as  a key  component.  The  Texas  program  was  unique  in  that  it  also  provided  a 
financial  incentive  of  $200  per  month  to  working  TANF  leavers.  Due  to  the  participation 
requirements  associated  with  TANF  receipt,  outreach  and  marketing  were  not  immediate 
priorities  in  the  designs  of  the  programs. 

Since  all  these  programs  served  unemployed  TANF  applicants  and  recipients,  the  target 
populations  were  relatively  similar.  The  demographic  characteristics  of  those  who  were  served 
are  displayed  in  Table  3.2.  In  most  of  these  sites,  only  about  half  the  sample  members  had  a 
high  school  diploma  or  a General  Educational  Development  (GED)  certificate  when  they 
entered  the  study.  Sample  members  in  these  sites  were,  on  average,  younger  than  30  years  old 
and  had  an  average  of  two  children,  and  those  children  were  young:  about  half  the  sample 
members  had  a youngest  child  age  2 or  younger.  In  most  of  the  sites  in  this  target  group,  large 
majorities  of  sample  members  had  two  years  or  less  of  prior  welfare  receipt  at  the  time  the  study 
began.  Although  these  programs  targeted  TANF  recipients  who  were  not  employed,  many 
sample  members  did  have  recent  work  experience,  as  the  table  shows:  during  the  quarter  before 
entering  the  study,  between  27  percent  and  49  percent  of  sample  members  in  these  sites  had 
been  employed  in  jobs  covered  by  the  unemployment  insurance  (UI)  system. 

Sample  members  in  this  target  group  left  the  first  job  they  held  after  random  assigmnent 
more  quickly  than  those  in  the  other  target  groups  covered  in  this  report  (Chapters  4 and  5). 
More  than  a third  of  the  control  group  left  their  first  job  by  the  end  of  the  first  quarter.  By  the 
third  quarter  after  the  study  began,  only  36  percent  were  still  working  at  their  original  employer. 
This  is  likely  explained  by  the  fact  that  sample  members  in  this  target  group  had  less  prior 
attachment  to  the  workforce  than  sample  members  in  the  other  target  groups. 

The  ERA  programs  serving  these  sample  members  experienced  some  implementation 
difficulties.  The  ERA  programs  in  Corpus  Christi  and  Los  Angeles  EJC  were  particularly  well 


1 For  complete  citations,  see  Appendix  E,  which  also  notes  where  the  reports  can  be  found  on  the  Web. 
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The  Employment  Retention  and  Advancement  Project 
Table  3.2 


Selected  Characteristics  of  ERA  Sample  Members  at  the  Time  of  Random  Assignment: 

Not  Employed  and  Receiving  TANF 


Characteristic"1 

Corpus 

Christi 

Texas 

Fort 

Worth 

Houston 

Los 

Angeles 

EJC 

Salem 

Female  (%) 

94.3 

96.1 

97.7 

91.0 

89.6 

Average  age  (years) 

28.7 

28.1 

27.8 

30.0 

29.5 

Race/ethnicity  (%) 

Flispanic 

73.9 

10.6 

27.5 

56.4 

15.4 

Black,  non-Hispanic 

8.1 

67.1 

61.9 

34.6 

2.7 

White,  non-Hispanic 

15.8 

21.5 

9.6 

5.6 

70.3 

Other 

2.2 

0.8 

1.1 

3.4 

11.6 

Average  number  of  minor  children 

1.9 

2.0 

2.1 

2.0 

1.7 

Age  of  youngest  child  (%) 

2 or  under 

49.6 

52.1 

51.5 

47.8 

41.4 

3 to  5 

21.0 

18.6 

21.5 

20.5 

23.2 

6 or  over 

29.4 

29.4 

27.1 

31.8 

35.4 

High  school  diploma/GED  certificate  or  higher  (%) 

48.2 

55.2 

44.9 

48.2 

68.5 

Total  prior  AFDC/TANF  receipt  (%) 

None 

38.8 

35.5 

38.2 

15.6 

41.3 

Less  than  2 years 

40.6 

49.8 

44.5 

41.1 

38.6 

2 years  or  more 

20.6 

14.7 

17.3 

43.4 

20.1 

Living  in  public  or  subsidized  housing  (%) 

16.5 

22.8 

23.2 

11.2 

17.8 

Employed  in  quarter  prior  to  random  assignmentb(%) 

48.6 

48.2 

43.2 

26.6 

45.3 

Employed  in  quarter  of  random  assignment b(%) 

49.3 

43.2 

39.2 

26.0 

42.8 

Employment  in  the  past  3 years  (%) 

Did  not  work 

14.5 

13.2 

14.6 

NA 

10.6 

Less  than  6 months 

18.2 

22.2 

21.1 

NA 

11.4 

7 to  12  months 

18.1 

17.5 

21.8 

NA 

16.0 

1 3 to  24  months 

21.3 

21.1 

19.8 

NA 

17.2 

More  than  24  months 

28.0 

26.1 

22.7 

NA 

44.8 

As  of  random  assignment  the  number  of 
hours  worked  per  week  among  employed  (%) 

1-19  hours 

28.5 

24.2 

24.6 

NA 

47.1 

20-31  hours 

40.9 

33.9 

36.8 

NA 

27.1 

32  or  more  hours 

30.7 

41.9 

38.6 

NA 

25.7 

As  of  random  assignment  the  hourly  wage 
among  employed  (%) 

Less  than  $5.15 

16.5 

6.5 

6.4 

1.8 

13.6 

$5.15  to  $6.99 

63.2 

53.2 

57.3 

25.8 

20.0 

$7.00  to  $9.99 

17.3 

27.4 

27.3 

43.7 

45.0 

$10.00  or  more 

3.0 

12.9 

9.1 

28.7 

21.4 

Sample  size 

1,727 

1,572 

2,032 

1,183 

1,504 

(continued) 
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Table  3.2  (continued) 


SOURCES:  Calculations  from  ERA  baseline  forms,  automated  records,  and  administrative  data. 

NOTES:  NA  = not  available.  In  this  case,  the  data  for  these  measures  were  not  collected. 
aStatistics  include  both  program  and  control  group  members. 
bThis  information  is  based  on  unemployment  insurance  (UI)  records. 


implemented,  and  the  ERA  program  in  Fort  Worth  made  significant  improvements  over  the 
course  of  the  study  period.  The  Houston  and  Salem  ERA  programs  both  experienced  a number 
of  implementation  issues.  All  the  programs,  except  for  Los  Angeles  EJC  (which  did  not  provide 
postemployment  services),  had  at  least  some  difficulty  providing  postemployment  services  to 
participants  who  found  work.  In  the  Texas  sites,  the  program  also  experienced  some  difficulties 
in  increasing  the  take-up  of  the  financial  incentive,  although  this  improved  over  time  in  the 
Corpus  Christi  and  Fort  Worth  sites. 
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Texas  ERA  Findings:  Corpus  Christi,  Fort  Worth,  and  Houston 


Texas  ERA  at  a Glance 

• Served  unemployed  TANF  applicants  and  recipients 

• Provided  a financial  incentive  for  those  leaving  TANF  for  work,  coupled  with  job 
search  assistance  and  postemployment  services  (including  reemployment  assistance  and 
visits  to  working  clients  at  their  employer’s  site) 

• Was  well  implemented  in  Corpus  Christi  but  experienced  some  operational  difficulties 
in  Fort  Worth  and,  particularly,  in  Houston 

• Compared  with  the  control  group,  differences  mostly  in  postemployment  services  and 
the  financial  incentive  (similar  preemployment  services) 

• Resulted  in  30  percent  taking  up  the  financial  incentive  in  Corpus  Christi  and  approx- 
imately 20  percent  doing  so  in  the  two  other  Texas  sites 

• Produced  consistent  impacts  in  employment  retention  and  advancement  outcomes  in 
Corpus  Christi,  less  consistent  impacts  in  Fort  Worth,  and  no  impacts  in  Houston 


Texas  ERA  Test:  Introduction 

The  Texas  ERA  program  was  designed  to  promote  job  placement,  retention,  and  ad- 
vancement for  TANF  applicants  and  recipients.  The  Texas  program  was  implemented  in 
Corpus  Christi,  Fort  Worth,  and  Houston  and  operated  from  2000  to  2004.  Results  for  each 
Texas  site  are  presented  separately  because  there  were  important  implementation  differences 
across  the  sites. 

Origins  of  the  Test 

The  ERA  program  in  Texas  was  explicitly  designed  to  improve  on  the  poor  perfor- 
mance of  past  retention  and  advancement  programs  — primarily  the  Post-Employment  Services 
Demonstration  (PESD)  programs  discussed  in  Chapter  1 — as  well  as  to  enhance  the  services 
provided  under  the  standard  TANF  employment  program.  The  program  design  grew  out  of  a 
concern  with  the  level  of  “recycling”  in  the  TANF  caseload  in  Texas.  TANF  grant  levels  in 
Texas  were  relatively  low  — the  maximum  monthly  grant  for  a family  of  three  was  $203  — 
and  while  Texas  had  a generous  earnings  disregard  (90  percent  of  earnings  were  disregarded  for 
four  months  when  calculating  the  size  of  TANF  grants),  most  individuals  in  Texas  who  took  a 
job  left  welfare  after  the  earned  income  disregard  period.  State  administrators  found  that  many 
of  these  individuals  leaving  welfare  worked  at  low  wages  with  poor  benefits  and  ended  up 
returning  to  the  rolls.  Thus,  the  ERA  program  was  designed  to  increase  employment  retention 
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and  wage  levels  for  those  leaving  welfare.  It  was  also  hoped  that  the  ERA  program  would 
increase  TANF  recipients’  participation  levels  in  preemployment  services  and  their  overall 
employment  levels. 

The  primary  innovation  of  the  Texas  ERA  program  was  a financial  incentive  designed 
to  encourage  work  and  employment  retention.  A stipend2  of  $200  per  month  was  provided  to 
participants  who  had  left  TANF  and  were  employed  for  a minimum  of  30  hours  per  week. 
Those  who  worked  15  hours  per  week  combined  with  an  education  and  training  activity  were 
also  eligible  for  the  stipend.  The  stipend  was  available  after  a four-month  period  during  which 
clients  received  an  earned  income  disregard  (an  amount  comparable  to  the  stipend,  because 
clients  received  almost  their  full  TANF  grant  in  addition  to  their  earnings),  and  there  was  a limit 
of  12  stipends.  Although  the  content  and  intensity  of  the  postemployment  services  varied  across 
the  sites,  the  Texas  program  provided  services  that  included  individualized  assistance  with  job- 
related  problems  and  support  services,  monitoring  of  job  perfonnance  and  issues  through 
regular  site  visits  to  employers,  reemployment  assistance  for  participants  who  lost  jobs,  and 
support  in  meeting  the  requirements  of  the  stipend.  Preemployment  job  search  assistance  was 
also  part  of  the  Texas  ERA  program,  but  this  service  was  similar  to  those  received  by  the 
control  group  through  the  standard  TANF  employment  program. 

Labor  Market  Context 

The  ERA  program  in  Texas  operated  in  three  diverse  sites,  although  all  had  relatively 
poor  economic  conditions  during  the  time  of  the  study.  Random  assignment  began  just  before 
the  national  recession  of  2001.  For  much  of  the  operational  period  of  the  Texas  ERA  program, 
the  labor  market  conditions  were  fairly  challenging,  with  growth  starting  in  the  last  few  years  of 
sample  member  follow-up. 

Corpus  Christi,  a midsize  city  located  on  the  Gulf  Coast,  has  an  economy  based  on  pe- 
troleum, shipping,  and  tourism.  In  2003,  the  unemployment  rate  was  6.7  percent  and  had 
increased  from  6.3  percent  in  1999,  when  the  study  began. 

Fort  Worth  is  the  sixth-largest  city  in  Texas.  Located  in  Tarrant  County,  it  is  a major 
mercantile,  commercial,  and  financial  center  providing  agribusiness  and  wholesale  services  to 
much  of  west  Texas.  In  December  1999,  the  unemployment  rate  was  2.8  percent,  but  it  in- 
creased to  6.4  percent  by  2003. 


2The  Texas  ERA  program  referred  to  these  monetary  benefits  as  “stipends.”  However,  throughout  MDRC 
publications  and  the  broader  workforce  development  literature,  they  are  also  referred  to  as  "financial  incen- 
tives” and  “wage  supplements,”  among  other  terms.  To  reflect  this  programmatic  choice,  the  discussion  of  the 
Texas  ERA  program  also  uses  “stipend.” 
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Houston  is  not  only  the  largest  city  in  Texas  but  also  the  fourth-most-populous  city  in 
the  nation.  Houston  is  regarded  as  a major  port  and  corporate  management  center,  and  the  city  is 
home  to  many  businesses,  including  petrochemicals;  medical  research  and  health  care  delivery; 
high  technology,  including  computer,  aerospace,  manufacturing,  and  distribution;  and  related 
service  industries.  As  of  December  1999,  the  unemployment  rate  was  4.2  percent,  but  it 
increased  to  6.9  percent  by  2003. 

Target  Population 

Individuals  were  assigned  to  the  ERA  or  control  group  immediately  following  an  eligi- 
bility or  recertification  interview  for  TANF.  (In  the  case  of  TANF  applicants,  random  assign- 
ment occurred  before  they  were  approved  for  cash  assistance.)  Only  those  applicants  and 
recipients  who  were  subsequently  approved  for  TANF  could  receive  ERA  services.  Thus,  a 
small  proportion  of  the  sample  (11  percent  to  16  percent  across  the  sites)  were  never  eligible  for 
program  services,  since  they  were  not  approved  to  receive  TANF.  This  diluted,  to  a small 
extent,  the  program-control  treatment  differential. 3 

There  were  some  differences  in  the  demographic  characteristics  of  the  sample  members 
across  the  three  sites  (Table  3.2).  Most  individuals  were  not  working  when  they  entered  the 
ERA  program.  Nearly  75  percent  of  the  Corpus  Christi  ERA  sample  members  are  Hispanic  — 
the  highest  percentage  of  all  the  ERA  sites  — in  contrast  with  Fort  Worth  and  Houston,  where 
the  majority  of  sample  members  are  black.  Over  half  the  sample  members  in  Corpus  Christi  and 
Houston  did  not  have  a high  school  diploma  or  GED  certificate  when  they  entered  the  study.  In 
fact,  the  Houston  sample  had  the  lowest  rate  of  educational  attainment  in  this  target  group. 

Significance  of  This  Test 

Overall,  the  Texas  ERA  test  examines  whether  job  search  assistance,  postemployment 
assistance  from  program  staff  (which  could  include  employer  site  visits  and  reemployment 
assistance),  and  a monthly  stipend  of  $200  for  TANF  leavers  working  at  least  30  hours  a week 
can  improve  employment  retention  and  advancement  for  TANF  applicants  and  recipients, 
compared  with  a preemployment  program  that  focused  on  quick  job  entry  with  limited  postem- 
ployment services.  Chart  3 . 1 highlights  important  treatment  differences  between  the  programs 
for  the  ERA  group  and  the  control  group  in  the  Texas  test. 


3ln  addition,  because  the  Texas  ERA  program  began  at  the  preemployment  phase  but  became  most  differ- 
ent after  sample  members  were  employed,  individuals  who  did  not  find  jobs  during  the  follow-up  period  had 
similar  experiences  in  the  ERA  and  control  groups,  thus  also  diluting  the  treatment  differential. 
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The  Employment  Retention  and  Advancement  Project 
Chart  3.1 

Program-Control  Group  Differences: 

Texas  ERA  Test  in  Corpus  Christi,  Fort  Worth,  and  Houston 


ERA  Group 

Control  Group 

Goals 

• To  promote  job  placements,  employment 
retention,  and  advancement  for  unemployed 
TANF  applicants  and  recipients 

• To  promote  job  placement  for  unemployed 
TANF  applicants  and  recipients 

Program 
resources  and 
structures 

• Except  for  financial  incentive,  funded  by  state 
TANF  program 

• Funded  by  state  TANF  program 

Staffing 

• Team-based  staff  structure  in  Corpus  Christi 
and  Fort  Worth 

• Staff  performance  measurement  system  in 
Corpus  Christi 

• Meetings  with  clients  at  employers'  sites  in 
Corpus  Christi  and  Fort  Worth 

• Generalist  staff  structure 

• No  performance  measurement  system 

• No  meetings  at  employers's  sites 

Staff-client 

engagement 

• Individualized  ongoing  assistance  from  staff 
after  leaving  TANF  for  work 

• Individualized  assistance  from  staff  only 
while  on  TANF  and  during  earned  income 
disregard  period 

Retention 

services 

• Offer  of  $200  per  month  stipend  for  former 
TANF  recipients  working  at  least  30  hours  per 
week,  after  4-month  earned  income  disregard 

• No  stipend 

• Regular  contact  with  working  clients  who  left 
TANF  and  received  stipend 

• Limited  contact  with  working  clients  and 
only  during  earned  income  disregard  period 

• Reemployment  services  in  Corpus  Christi  and 
Fort  Worth 

• No  reemployment  services 

Advancement 

services 

• Discussions  with  workers  and  their  employers 
about  advancement  options  in  Corpus  Christi 
and  Fort  Worth 

• No  advancement  services 

Employer 

linkages 

• Site  visits  to  employer 

• Limited  contact  with  employers 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 
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Texas  ERA  Program:  Features  and  Client  Experiences 

This  section  discusses  the  key  features  of  the  Texas  ERA  program,  highlighting  impor- 
tant implementation  differences  across  the  three  sites,  and  describes  client  experiences  in  the 
program  as  well. 

Program  Resources  and  Organization 

The  Texas  ERA  services  were  primarily  funded  by  TANF  resources.  The  exception  is 
the  Texas  ERA  stipend,  which  was  supported  by  federal  Aid  to  Families  with  Dependent 
Children  (AFDC)  sanctioning  resettlement  funds.4  As  a result  of  using  this  financing  source  for 
the  stipend,  TANF  regulations  on  the  definition  of  “assistance”  did  not  apply  to  the  stipend. 
Payments  that  are  categorized  as  assistance  are  subject  to  certain  TANF  rules,  primarily  a five- 
year  lifetime  limit  on  receipt. 

The  state’s  Department  of  Human  Services  (DHS),  in  coordination  with  the  Texas 
Workforce  Commission  (TWC),  designed  the  Texas  ERA  program.  At  the  local  level,  all 
employment  services  for  TANF  recipients  were  coordinated  by  local  workforce  development 
boards,  under  the  guidance  of  TWC.  In  each  of  the  sites,  the  local  workforce  development 
boards  contracted  with  nonprofit  organizations  to  operate  the  ERA  program.  Each  site  had  some 
discretion  in  how  it  structured  the  ERA  program,  although  each  followed  the  same  general 
model. 


The  Coastal  Bend  Workforce  Development  Board  oversaw  the  ERA  program  in  Cor- 
pus Christi.  In  turn,  it  contracted  with  a local  nonprofit  organization,  Workforce- 1,  to  operate 
the  ERA  program.  In  Corpus  Christi,  a strong  management  team  was  in  place  during  the  early 
phases  of  the  project  and  was  responsible  for  developing  the  initial  ERA  program,  with  new 
management  taking  over  in  2002  when  this  team  moved  on  to  other  opportunities.  In  Fort 
Worth,  the  Workforce  Development  Board  contracted  with  a nonprofit  organization,  the 
Women’s  Center,  for  some  ERA  services  (primarily  ongoing  staff-client  engagement  and 
postemployment  services),  while  also  using  the  provider  of  the  control  group  services  for  the 
preemployment  ERA  services.  While  the  program  experienced  several  initial  implementation 
issues,  the  introduction  of  a new  program  manager  in  fall  2002  resulted  in  many  program 
improvements.  In  Houston,  the  ERA  program  was  operated  by  HoustonWorks  USA  (a  local 
nonprofit  agency  board),  under  contract  to  the  Houston-Galveston  Area  Council  (the  local 


4A  state  that  failed  to  meet  some  AFDC  performance  criteria  had  a part  of  its  grant  rescinded  or  reduced; 
that  is,  the  state  was  sanctioned.  These  extra  monies  were  often  distributed  to  nonsanctioned  states  (or  "high- 
performer”  states)  for  program  improvements.  These  monies  were  not  subject  to  the  same  limitations  on  types 
of  expenditures  as  standard  AFDC  grants  to  states  were. 
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workforce  investment  board).  Nonprofit  organizations  were  also  the  providers  of  the  preem- 
ployment services  for  control  group  members. 

Staffing 

Texas  ERA  services  were  provided  primarily  by  program  staff  who  were  employed  by 
the  nonprofit  agency  operating  the  ERA  program  in  each  site.  While  staff  responsibilities  in  the 
ERA  program  evolved  over  time,  all  sites  ended  up  using  a specialized  service  delivery  ap- 
proach — with  one  set  of  staff  being  responsible  for  preemployment  activities  (primarily 
assessment  of  clients  and  monitoring  their  participation);  another  set  of  staff  responsible  for  job 
search,  job  development,  and  reemployment  assistance  (in  Corpus  Christi  and  Fort  Worth  only); 
and  another  set  responsible  for  postemployment  services  (monitoring  individuals’  employment 
status,  issuing  stipends,  assisting  with  job-related  issues).  In  all  the  sites,  DHS  staff — not  ERA 
program  staff — were  responsible  for  TANF  eligibility  and  benefit  issues. 

There  were,  however,  important  distinctions  in  staffing  across  the  sites.  Corpus  Christi 
moved  most  quickly  to  establish  its  specialized  staffing  structure.  Fort  Worth  used  staff  from 
two  agencies  to  provide  different  services  to  ERA  participants  for  much  of  the  study  period;  one 
agency  focused  on  client  assessment  and  monitoring,  and  the  other  focused  on  job  search 
activities.  Particularly  during  the  early  phases  of  the  study,  there  were  some  coordination  issues 
between  these  two  agencies,  which  affected  the  client  flow  through  activities.  Houston  used  a 
generalized  staffing  approach  for  much  of  the  study  period  but  moved  to  a specialized  approach 
over  time. 

The  number  of  staff  in  the  programs  varied.  Corpus  Christi  generally  had  the  largest 
staff,  at  one  point  having  12  program  staff  as  well  as  other  administrative  staff  and  a workshop 
facilitator.  In  Fort  Worth  and  Houston  there  were  fewer  staff  (six  to  eight).  All  the  sites  went 
through  some  period  over  the  course  of  the  study  when  they  were  short-staffed  and  caseloads 
were  higher  than  intended. 

Corpus  Christi  was  the  only  Texas  site  to  develop  a comprehensive  system  for  measur- 
ing staff  performance.  The  performance  measures  evolved  somewhat  over  time,  and  both 
management  and  line  staff  took  them  seriously.  The  measures  focused  on  key  goals,  including 
participation  levels,  job  placements,  retention  rates,  wage  increases,  employer  site  visits,  and 
payment  of  stipends.  The  other  sites  generally  did  not  have  specific  measures  for  staff  perfor- 
mance — particularly  related  to  employment  retention  and  advancement  issues.  For  example, 
Houston  had  some  job  placement  measures  for  staff  but  none  that  were  related  to  postemploy- 
ment services  or  stipend  receipt. 

Staff  varied  in  their  previous  employment  experiences.  Some  had  a history  of  working 
with  welfare-to-work  programs,  particularly  in  the  standard  TANF  employment  program  (the 
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services  received  by  the  control  group).  Others  did  not  have  experience  in  social  services  but 
brought  experience  from  related  fields,  such  as  employment  placement  or  temporary  job 
agencies.  DHS  held  annual  training  sessions  for  ERA  staff  in  all  the  sites,  covering  a range  of 
issues  related  to  the  program,  including  marketing  financial  incentives,  developing  career 
ladders,  and  identifying  strategies  for  preventing  job  loss  and  promoting  advancement. 

The  Texas  ERA  program  generally  did  not  provide  significant  flexibility  for  program 
group  members  in  terms  of  meeting  hours.  The  programs  did  not  include  extended  hours, 
although,  by  the  end  of  the  study  period,  all  the  Texas  sites  ran  evening  and  weekend  support 
groups  for  individuals  receiving  the  stipend.  In  Corpus  Christi  and  Fort  Worth,  staff  could  also 
meet  with  clients  outside  the  office,  particularly  at  their  workplaces. 

Staff-Client  Engagement 

The  Texas  ERA  and  control  group  preemployment  programs  were  mandatory  for  indi- 
viduals while  they  were  receiving  TANF,  meaning  that  clients  faced  a possible  reduction  or 
termination  of  TANF  benefits  for  noncompliance  with  program  rules.  Individuals  in  both 
programs  were  not  required  to  participate  in  program  activities  after  they  left  TANF,  although 
there  were  requirements,  in  addition  to  working,  for  receiving  the  stipend. 

Because  the  Texas  ERA  program  was  mandatory  at  the  outset,  outreach  and  marketing 
of  most  services  were  not  a major  component  of  the  program.  However,  all  sites  found  that  the 
“take-up”  of  the  stipend  was  much  lower  than  expected,  leading  them  to  develop  marketing 
strategies  focused  on  the  financial  incentive. 

Corpus  Christi  moved  more  quickly  than  the  two  other  ERA  sites  in  Texas  to  develop  a 
strong  marketing  effort  for  the  $200  monthly  stipend.  Marketing  strategies  included  (1)  telling 
individuals  about  the  stipend  and  strongly  encouraging  them  to  take  advantage  of  the  stipend  at 
all  junctures  in  the  program  — immediately  after  they  were  assigned  to  the  program,  at  assess- 
ment, during  job  search  activities,  when  they  found  a job,  and  when  they  had  finished  the 
earnings  disregard  period;  (2)  hanging  posters  and  displaying  fliers  throughout  the  ERA  offices 
informing  participants  of  this  benefit;  (3)  developing  strategies  to  strengthen  the  appeal  of  the 
payments  — for  example,  calling  the  incentive  an  employment  bonus  and  emphasizing  the  total 
amount  of  the  payment  ($2,400  over  one  year);  and  (4)  creating  posters  and  videos  that  told 
“success  stories”  of  participants  who  received  the  stipend  and  how  they  used  the  resource. 

In  Fort  Worth,  there  were  limited  efforts  to  market  the  stipend  during  the  initial  stages 
of  the  program.  However,  when  the  new  management  team  was  put  in  place,  staff  did  make 
improved  efforts  in  this  area  using  many  of  the  strategies  developed  in  Corpus  Christi.  Houston 
did  not  market  the  financial  incentive  in  a concerted  manner  until  late  into  the  study  period,  and 
it  also  drew  on  the  efforts  developed  in  Corpus  Christi. 
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The  client  assessments  in  the  Texas  ERA  program  generally  focused  on  reviewing  a 
client’s  education  and  work  history  and  addressing  support  service  needs.  In  Corpus  Christi, 
there  was  some  focus  on  identifying  long-tenn  career  goals  and  pathways  as  part  of  the  assess- 
ment, although  an  overall  focus  on  quick  employment  remained.  In  Fort  Worth,  a multistep 
assessment  process  involving  different  staff  was  put  in  place  initially,  but  the  process  had  the 
effect  of  limiting  the  number  of  individuals  who  received  other  program  services  and  moved  to 
employment.  Eventually  a more  streamlined  assessment  was  developed  and  conducted  as  part 
of  the  first  week  of  the  job  search  workshop.  Houston  also  included  an  additional  in-depth 
assessment  for  many  participants  (which  involved  an  extensive  interview  with  a staff  person 
who  was  a licensed  psychologist)  throughout  the  study  period. 

The  stalf-client  relationship  in  the  Texas  ERA  program  started  during  the  initial  preem- 
ployment services  and  continued  after  individuals  found  employment,  both  during  the  earnings 
disregard  period  and  once  they  started  receiving  a stipend.  In  contrast,  interactions  between  staff 
and  control  group  members  were  less  frequent  for  individuals  once  they  were  working,  and 
interactions  did  not  continue  after  the  disregard  period  ended. 

Reflecting  this  difference  in  service  availability,  program  group  members  in  Corpus 
Christi  and  Houston  reported  significantly  more  contact  with  staff  from  an  employment  pro- 
gram, when  surveyed  12  months  after  study  entry,  than  control  group  members  did.  Table  3.3 
shows  that  over  60  percent  of  the  Corpus  Christi  program  group  had  been  contacted  by  program 
staff  within  this  period  (13  percentage  points  more  than  the  control  group),  and  program  group 
members  were  also  more  likely  to  be  in  contact  with  staff  at  the  time  of  the  12-month  survey. 
Similarly,  in  Fort  Worth,  63  percent  of  the  program  group  were  contacted  within  the  first  12 
months  — a rate  18  percentage  points  higher  than  the  control  group  rate.  In  Fort  Worth,  the 
program  group  had  an  average  of  14  contacts  in  the  first  12  months,  compared  with  an  average 
of  5 contacts  for  the  control  group.  The  ongoing  nature  of  the  contact  was  also  much  more 
pronounced  in  Fort  Worth;  Table  3.4  shows  that  nearly  30  percent  of  the  program  group  were  in 
contact  at  the  end  of  one  year,  compared  with  9 percent  of  the  control  group.  In  contrast,  across 
the  range  of  measures  of  staff-client  contact  included  in  the  survey,  the  ERA  program  in 
Houston  did  not  produce  any  statistically  significant  effects. 

Initial  Job  Preparation  and  Placement  Services 

Job  placement  was  one  of  the  primary  goals  in  the  Texas  ERA  program,  and  the 
preemployment  services  were  focused  on  this  objective.  As  the  first  step  in  the  program, 
participants  generally  attended  a one-week  job  search  workshop,  where  a range  of  job  search- 
related  activities  — including  job  search  resources,  resume  development,  and  interviewing 
techniques  — were  covered.  Once  the  class  was  completed,  clients  were  required  to  make  and 
document  a specified  number  of  job  search  contacts  per  week  for  a three-  to  five- week  period. 
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The  Employment  Retention  and  Advancement  Project 
Table  3.3 

Summary  of  Impacts: 


Corpus  Christi 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

60.2 

47.7 

12.5  ** 

0.041 

Average  number  of  contacts  with  staff 

6.6 

6.4 

0.3 

0.889 

Talked  with  staff  in  past  4 weeks  (%) 

27.2 

16.5 

10.7  ** 

0.036 

Retention  services 

Participated  in  a job  search  activity  (%) 

62.0 

58.2 

3.8 

0.531 

Received  help  dealing  with  problems  on  the  job  (%) 

8.4 

1.5 

6.9  *** 

0.009 

Advancement  services 

Received  help  with  career  assessment  (%) 

17.8 

7.4 

10.4  ** 

0.010 

Received  help  finding  a better  job  while  working  (%) 

13.6 

3.9 

q y *** 

0.006 

Participated  in  an  education/training  activity3  (%) 

30.6 

22.7 

7.9 

0.150 

Received  any  help  with  retention/advancement1’  (%) 

28.5 

14.0 

14  6 *** 

0.003 

Sample  size  (total  = 290) 

141 

149 

Cumulative  economic  outcomes  (Years  1-4) 

Employment  measures 

Ever  employed  (%) 

88.4 

89.3 

-0.9 

0.535 

Average  quarterly  employment  (%) 

51.7 

48.0 

3 7 ** 

0.012 

Had  employment  spell  of  at  least  4 quarters  (%) 

63.4 

56.9 

6 5 *** 

0.003 

Length  of  longest  employment  spell,  in  quarters 

6.4 

6.0 

q 4 ** 

0.044 

Length  of  longest  unemployment  spell,  in  quarters 

6.1 

6.6 

-0.4  * 

0.057 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

5,011 

4,371 

640  *** 

0.007 

Quarters  with  earnings  of  $3,500  or  more  (%) 

12.8 

10.9 

1.9  ** 

0.045 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

44.7 

47.4 

-2.7 

0.219 

Earnings  decreased  by  more  than  $250 

15.8 

15.7 

0.1 

0.934 

Earnings  changed  by  less  than  $250 

7.1 

7.1 

0.0 

0.987 

Earnings  increased  by  $250  or  more 

32.5 

29.8 

2.6 

0.224 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

474 

505 

-31 

0.171 

Average  annual  food  stamps  received  ($) 

2,589 

2,723 

-134  ** 

0.050 

Average  annual  income3  ($) 

8,074 

7,599 

475  ** 

0.040 

Sample  size  (total  = 1,727) 

870 

857 

(continued) 
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Table  3.3  (continued) 


SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

“This  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


The  ERA  program  in  Fort  Worth  and  Houston  produced  increases  in  the  level  of  partic- 
ipation in  job  search  activities.  In  Fort  Worth,  the  ERA  program  increased  the  proportion  of 
people  who  participated  in  group  job  search:  63  percent  of  the  program  group  participated  in  a 
group  job  search/job  club  in  the  first  12  months,  compared  with  49  percent  of  the  control  group 
(Appendix  Table  A.5).  But  levels  of  participation  in  any  type  of  job  search,  which  includes 
individual  job  search  as  well,  were  the  same  for  both  groups.  In  Houston,  the  ERA  program 
produced  increases  in  participation  in  any  type  of  job  search:  80  percent  of  program  group 
members  in  Houston  participated  in  some  type  of  job  search  activity  in  the  first  12  months  — an 
increase  of  10  percentage  points  over  the  control  group  (Table  3.5).  Similar  proportion  of 
program  and  control  group  members  in  Corpus  Christi  participated  in  job  search  activities: 
about  60  percent  (Table  3.3). 

Retention  Sen/ices 

For  employed  participants,  the  goal  of  the  Texas  ERA  program  was  employment  reten- 
tion and  advancement  in  the  labor  market,  with  both  individualized  assistance  from  program 
staff  and  financial  incentives  provided  to  assist  with  this  goal.  Individuals  in  the  ERA  program 
who  found  jobs  were  required  to  work  for  four  months  (while  receiving  an  earned  income 
disregard)  prior  to  receiving  the  stipend.  To  receive  the  stipend,  program  group  members  had  to 
work  the  required  number  of  hours,  submit  necessary  documentation,  and  attend  some  type  of 
employment-related  activity  each  month.  Individualized  assistance  from  program  staff  on  job- 
related  issues  was  provided  to  working  clients  throughout  the  earned  income  disregard  and 
stipend  period. 
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The  Employment  Retention  and  Advancement  Project 
Table  3.4 

Summary  of  Impacts: 

Fort  Worth 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

63.0 

44.9 

18.1  ** 

0.014 

Average  number  of  contacts  with  staff 

14.0 

5.2 

g 9 *** 

0.007 

Talked  with  staff  in  past  4 weeks  (%) 

29.7 

9.0 

2Q  7 *** 

0.001 

Retention  services 

Participated  in  a job  search  activity  (%) 

77.4 

72.7 

4.6 

0.500 

Received  help  dealing  with  problems  on  the  job  (%) 

5.8 

1.7 

4.1 

0.177 

Advancement  services 

Received  help  with  career  assessment  (%) 

20.9 

6.0 

14  9 *** 

0.005 

Received  help  finding  a better  job  while  working  (%) 

13.5 

2.7 

10.7  ** 

0.012 

Participated  in  an  education/training  activity3  (%) 

38.7 

29.5 

9.2 

0.193 

Received  any  help  with  retention/advancement'’  (%) 

31.0 

10.9 

20.1  *** 

0.002 

Sample  size  (total  = 188) 

92 

96 

Cumulative  economic  outcomes  (Years  1-4) 

Employment  measures 

Ever  employed  (%) 

86.9 

85.0 

2.0 

0.226 

Average  quarterly  employment  (%) 

48.5 

46.9 

1.6 

0.274 

Had  employment  spell  of  at  least  4 quarters  (%) 

61.3 

58.5 

2.8 

0.214 

Length  of  longest  employment  spell,  in  quarters 

6.1 

5.8 

0.3 

0.153 

Length  of  longest  unemployment  spell,  in  quarters 

6.6 

6.9 

-0.3 

0.160 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

5,706 

5,256 

449 

0.148 

Quarters  with  earnings  of  $3,500  or  more  (%) 

17.1 

15.5 

1.7 

0.154 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

49.7 

49.5 

0.2 

0.932 

Earnings  decreased  by  more  than  $250 

14.7 

14.1 

0.7 

0.712 

Earnings  changed  by  less  than  $250 

5.1 

4.9 

0.2 

0.855 

Earnings  increased  by  $250  or  more 

30.3 

31.5 

-1.2 

0.601 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

606 

621 

-15 

0.570 

Average  annual  food  stamps  received  ($) 

2,690 

2,528 

162  ** 

0.025 

Average  annual  income3  ($) 

9,002 

8,406 

596  ** 

0.043 

Sample  size  (total  = 1,572) 

784 

788 

(continued) 
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Table  3.4  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

This  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

“This  measure  represents  the  sum  of  UI -reported  earnings,  TANF,  and  food  stamps. 


Program  staff  were  generally  required  to  have  monthly  contact  with  working  partici- 
pants during  the  earnings  disregard  period.  In  order  to  determine  continued  eligibility  for  TANF 
and  the  earned  income  disregard,  program  staff  were  required  to  verily  monthly  the  hours  and 
wages  of  employment  through  these  contacts,  but  they  also  used  the  opportunity  to  address  any 
job-related  issues.  These  contacts  typically  took  place  by  telephone,  and  participants  faxed  any 
necessary  paperwork.  In  addition  to  these  contacts,  staff  typically  met  in  person  with  ERA 
participants  once  during  the  earned  income  disregard  period,  to  encourage  individuals  to  take 
advantage  of  the  stipend  when  they  became  eligible. 

Individuals  receiving  the  stipend  were  required  to  attend  some  type  of  employment- 
related  activity  each  month.  The  ERA  program  in  all  three  sites  was  generally  flexible  in  what  it 
allowed  to  count  toward  this  requirement  — including  training  at  work,  education  and  training 
programs,  various  support  groups,  or  other  activities  in  the  community.  By  the  end  of  the  study 
period,  all  the  sites  sponsored  their  own  support  groups  that  program  group  members  could 
attend  to  meet  the  requirement.  Program  staff  also  provided  a monthly  calendar  of  both  ERA- 
operated  support  groups  and  other  community  events  that  could  be  used  to  fulfill  this  require- 
ment. In  Corpus  Christi  and  Fort  Worth,  staff  handed  out  stipend  checks  at  a monthly  work- 
shop, which  allowed  individuals  to  get  their  stipend  check  and  meet  the  attendance  requirement 
at  the  same  time. 

Once  individuals  started  receiving  the  stipend,  postemployment  staff  were  expected  to 
continue  to  make  at  least  monthly  contact  with  individuals  on  their  caseload.  Staff  focused  on 
employment  retention  and  job-related  problems  while  also  ensuring  that  individuals  met  the 


59 


The  Employment  Retention  and  Advancement  Project 
Table  3.5 

Summary  of  Impacts: 

Houston 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

59.5 

58.4 

1.1 

0.853 

Average  number  of  contacts  with  staff 

9.7 

6.1 

3.6 

0.118 

Talked  with  staff  in  past  4 weeks  (%) 

21.0 

20.1 

1.0 

0.836 

Retention  services 

Participated  in  a job  search  activity  (%) 

79.6 

69.1 

10.4  ** 

0.038 

Received  help  dealing  with  problems  on  the  job  (%) 

11.7 

6.4 

5.3 

0.111 

Advancement  services 

Received  help  with  career  assessment  (%) 

12.4 

11.2 

1.1 

0.756 

Received  help  finding  a better  job  while  working  (%) 

10.0 

5.4 

4.6 

0.135 

Participated  in  an  education/training  activity3  (%) 

35.2 

34.9 

0.3 

0.960 

Received  any  help  with  retention/advancement0  (%) 

24.0 

18.3 

5.7 

0.233 

Sample  size  (total  = 297) 

150 

147 

Cumulative  economic  outcomes  (Years  1-4) 

Employment  measures 

Ever  employed  (%) 

82.0 

83.2 

-1.2 

0.461 

Average  quarterly  employment  (%) 

43.4 

43.3 

0.1 

0.963 

Had  employment  spell  of  at  least  4 quarters  (%) 

54.9 

53.4 

1.6 

0.443 

Length  of  longest  employment  spell,  in  quarters 

5.3 

5.4 

-0.1 

0.656 

Length  of  longest  unemployment  spell,  in  quarters 

7.5 

7.5 

0.0 

0.990 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

4,722 

4,777 

-56 

0.819 

Quarters  with  earnings  of  $3,500  or  more  (%) 

12.9 

13.8 

-0.9 

0.303 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

53.9 

54.9 

-1.0 

0.610 

Earnings  decreased  by  more  than  $250 

13.8 

15.2 

-1.4 

0.351 

Earnings  changed  by  less  than  $250 

5.1 

3.5 

1.6  * 

0.078 

Earnings  increased  by  $250  or  more 

27.0 

26.1 

0.9 

0.631 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

759 

703 

56  ** 

0.033 

Average  annual  food  stamps  received  ($) 

2,863 

2,822 

41 

0.538 

Average  annual  income3  ($) 

8,344 

8,302 

41 

0.860 

Sample  size  (total  = 2,032) 

1,009 

1,023 

(continued) 
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Table  3.5  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

“This  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


requirements  to  receive  the  stipend.  Staff  also  addressed  issues  related  to  work  supports  and 
social  service  referrals  as  needed. 

In  order  to  more  effectively  address  both  retention  and  advancement  issues,  Corpus 
Christi  and  Fort  Worth  staff  conducted  many  of  their  monthly  meetings  with  individuals 
receiving  the  stipend  at  the  work  site,  although  Fort  Worth  did  not  begin  employer  visits  until 
late  in  the  study  period.  Office  visits  were  difficult  when  participants  were  working.  In  addition, 
staff  found  in-person  meetings  more  valuable  than  phone  contacts.  These  short  meetings 
generally  included  discussions  with  both  the  employee  and  the  employer  about  job-related 
issues,  job  performance,  and  advancement  options. 

Although  program  staff  knew  that  job  loss  was  a common  problem  for  the  ERA  popula- 
tion, they  were  still  somewhat  surprised  by  its  magnitude.  Sometimes  participants  would  inform 
program  staff  if  they  were  going  to  lose  their  job  or  quit,  but  other  times  staff  did  not  find  out 
about  an  employment  change  until  they  attempted  their  regular  monthly  contact.  Because  of 
this,  all  the  sites  tried  to  strengthen  their  reemployment  services.  Corpus  Christi  and  Fort  Worth 
had  goals  of  reemploying  ERA  participants  within  two  weeks  of  finding  out  about  job  loss.  This 
assistance  was  provided  regardless  of  whether  the  individual  was  on  TANF  or  not.  Fort  Worth 
also  designated  a program  staff  member  to  address  all  cases  that  needed  reemployment  services. 

Overall,  Corpus  Christi  implemented  the  postemployment  component  of  the  ERA  pro- 
gram most  smoothly,  with  the  other  sites  adopting  some  of  their  strategies  over  time.  Program 
practices  in  Corpus  Christi  included  designating  program  staff  to  work  only  with  individuals 
receiving  stipends,  regular  site  visits  to  employers,  in-house  support  groups  to  meet  the  stipend 
participation  requirement,  and  reemployment  services.  These  practices  were  developed  by  the 
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strong  management  team  that  the  program  had  in  place  during  the  early  phases  of  the  project. 
The  ERA  program  in  Fort  Worth  struggled  during  the  early  portion  of  the  study  period,  but  it 
made  significant  improvements  when  a new  manager  was  hired,  including  establishing  a 
number  of  new  practices,  such  as  regular  site  visits  to  employers.  Houston  moved  the  most 
slowly  to  get  key  components  of  the  program  off  the  ground  — particularly  postemployment 
services  — and  it  focused  on  these  only  near  the  end  of  the  study  period. 

Of  the  three  sites  that  implemented  the  Texas  ERA  program,  stipend  receipt  rates  were 
highest  in  Corpus  Christi,  with  30  percent  of  all  program  group  members  (including  those  who 
found  jobs  as  well  as  those  who  did  not)  receiving  a stipend  over  the  course  of  program  opera- 
tions. The  stipend  receipt  rate  was  lower  in  Fort  Worth  and  Houston,  with  approximately  20 
percent  of  program  group  members  ever  receiving  a stipend.  In  both  Corpus  Christi  and  Fort 
Worth,  almost  half  of  those  who  received  a stipend  received  1 1 or  more  stipends.  However,  in 
Houston,  the  stipend  was  received  much  less  consistently  than  in  the  other  sites  — close  to  two- 
thirds  of  those  who  received  a stipend  received  six  or  fewer  stipends,  and  only  9 percent 
received  1 1 or  more  stipends. 

One  explanation  for  why  stipend  receipt  rates  were  not  higher  concerns  eligibility  crite- 
ria: some  individuals  in  the  program  group  did  not  qualify  for  the  stipend  because  they  were  not 
approved  to  receive  TANF,  they  did  not  find  jobs,  or  they  did  not  work  longer  than  the  four- 
month  earned  income  disregard  period.  Because  of  these  steps,  a relatively  long  period  of  time 
elapsed  between  when  individuals  enrolled  in  the  program  and  they  received  their  first  stipend: 
about  17  months,  on  average  (and  somewhat  shorter  in  Fort  Worth).  Furthermore,  while 
individuals  who  received  the  entire  earned  income  disregard  generally  became  eligible  for  the 
stipend,  they  may  not  have  received  it  if  they  did  not  work  enough  hours,  submit  the  required 
documentation,  or  attend  the  required  activity  in  a given  month.  Overall,  among  those  who  were 
eligible  because  they  worked  at  least  four  months,  approximately  55  percent  received  a stipend 
in  Corpus  Christi,  and  40  percent  received  one  in  Fort  Worth  and  Houston. 

As  discussed  above,  all  the  sites  developed  a solid  marketing  effort  for  the  ERA  sti- 
pends, particularly  over  time,  with  Corpus  Christi  moving  most  quickly  to  develop  marketing 
materials  and  strategies.  However,  with  only  about  half  of  those  who  were  eligible  receiving 
stipends,  there  was  still  clearly  room  for  improvement.  Based  on  phone  interviews  with  ERA 
participants,  reasons  for  not  accessing  the  stipend  among  those  who  were  eligible  included  not 
attending  the  required  monthly  employment  activity,  a desire  to  discontinue  involvement  with  a 
government  program,  and  a lack  of  knowledge  or  understanding  of  the  stipend  requirements.  In 
addition,  some  participants  lost  jobs  before  they  became  eligible  for  the  stipend. 

Overall,  data  from  the  12-month  survey  suggest  that  the  stipend  may  have  been  the 
primary  difference  in  retention  services  for  the  program  and  control  groups.  However,  there 
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were  some  limited  exceptions.  In  Corpus  Christi,  for  example,  the  program  group  members 
were  more  likely  than  control  group  members  to  receive  help  dealing  with  problems  on  the  job 
(Table  3.3)  and  to  receive  help  with  support  services  (Appendix  Table  A.1). 

Advancement  Services 

For  those  individuals  who  became  relatively  stable  in  their  jobs,  Texas  ERA  staff  also 
worked  with  clients  on  advancement  options  — primarily  in  terms  of  the  next  job  at  the 
employer  or  a job  change.  Like  retention  services,  these  services  were  put  in  place  most  quickly 
in  Corpus  Christi,  were  developed  over  time  in  Fort  Worth,  and  were  not  addressed  in  Houston 
until  late  in  the  study  period.  In  Corpus  Christi  and  Fort  Worth,  program  staff  made  efforts  to 
discuss  participants’  career  paths,  views  of  their  ideal  job,  and  what  was  needed  to  obtain  an 
ideal  job.  In  addition,  ERA  staff  in  Corpus  Christi  and  Fort  Worth  visited  their  clients’  work 
sites.  These  employer  visits  included  examining  potential  advancement  opportunities  within  the 
workplace  and  strategies  for  moving  to  a better  job,  such  as  obtaining  more  training  or  taking 
more  initiative  on  the  job.  ERA  staff  also  worked  with  employers  to  identify  what  workers 
needed  to  do  to  advance  in  their  jobs  and  what  criteria  were  used  to  evaluate  employees  and 
give  raises.  Although  the  ERA  program  in  Houston  intended  to  support  advancement  strategies, 
its  actual  implementation  offered  only  limited  services  in  this  area. 

Overall,  participants  in  the  Corpus  Christi  site  reported  significantly  more  assistance 
with  employment  advancement,  compared  with  control  group  members,  particularly  in  terms  of 
finding  a better  job  while  working  and  receiving  a career  assessment  (Table  3.3).  Similarly, 
ERA  participants  in  Fort  Worth  reported  significantly  more  assistance  with  advancement  issues 
than  individuals  assigned  to  the  control  group.  The  ERA  program  in  Fort  Worth  increased  the 
proportion  of  individuals  who  received  assistance  finding  a better  job  while  working  and 
completing  a career  assessment  (Appendix  Table  A.5).  In  Houston,  there  was  no  increase  in 
participation  in  advancement  services  relative  to  the  control  group  (Table  3.5). 

While  individuals  could  work  part  time,  participate  in  education  and  training,  and  still 
receive  a stipend,  this  was  a rarefy  used  option  in  any  of  the  Texas  ERA  sites.  As  a result,  there 
was  no  increase  in  the  use  of  education  and  training  programs  compared  with  the  control 
condition  (Tables  3.3,  3.4,  and  3.5). 

Overall,  about  29  percent  of  the  program  group  in  Corpus  Christi  participated  in  any  re- 
tention or  advancement  services  in  the  12  months  following  study  entry  — a rate  double  that  of 
the  control  group  (Table  3.3).  In  Fort  Worth,  this  rate  was  31  percent  of  the  program  group  — 
nearly  triple  the  rate  of  participation  in  these  services  by  the  control  group  (Table  3.4).  In 
Houston,  there  was  no  increase  in  overall  participation  in  retention  or  advancement  services 
relative  to  the  control  group  (Table  3.5). 
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Employer  Linkages 

Like  other  program  services,  the  connection  between  the  Texas  ERA  program  and  em- 
ployers was  strongest  in  Corpus  Christi,  gained  strength  in  Fort  Worth,  and  was  underdeveloped 
in  Houston.  In  Corpus  Christi,  for  most  of  the  study  period,  postemployment  staff  conducted 
employer  site  visits  at  frequent  intervals  after  an  individual  found  a job  and,  later,  moved  to  the 
goal  of  monthly  employer  site  visits  whenever  possible.  As  a result,  nearly  1 8 percent  of  the 
ERA  program  group  in  Corpus  Christi  reported  that  their  employer  had  been  contacted  by  a 
program  staff  member  — a rate  significantly  greater  than  the  2 percent  of  the  control  group  that 
reported  this  (Appendix  Table  A.l).  In  addition,  ERA  job  developers  in  Corpus  Christi  tried  to 
solicit  new  companies  to  provide  opportunities  to  program  group  members,  particularly  compa- 
nies that  currently  employed  program  group  members.  Stalf  also  attempted  to  identify  new 
employment  opportunities  at  the  employers  they  were  visiting  when  they  checked  on  clients. 

Fort  Worth  started  conducting  employer  site  visits  similar  to  those  in  Corpus  Christi  lat- 
er in  the  study  period.  In  this  site,  8 percent  of  the  program  group  reported  that  an  ERA  staff 
member  had  contacted  their  employer,  compared  with  1 percent  of  the  control  group  (Appendix 
Table  A.5).  No  specific  job  development  was  done  for  the  ERA  program  in  Fort  Worth. 

Due  to  local  administrative  issues,  the  ERA  program  in  Houston  had  difficulty  imple- 
menting employer  site  visits  for  working  individuals  and  dedicating  job  development  staff  to  the 
ERA  program.  As  a result,  the  program  had  limited  linkages  to  employers,  and  there  was  no 
increase,  relative  to  the  control  group,  in  the  percentage  of  program  group  members  reporting 
that  ERA  staff  had  contact  with  their  employer  (Appendix  Table  A.  8). 

Texas  ERA  Program:  Economic  Impacts 

As  discussed,  there  were  important  differences  in  program  services  and  implementation 
in  the  three  Texas  sites.  Because  of  these  differences,  the  economic  impact  results  for  each  of 
the  Texas  sites  are  presented  separately. 

Corpus  Christi  ERA  Site:  Economic  Impacts 

Economic  impacts  of  the  Corpus  Christi  ERA  program  can  be  summarized  as  follows: 

• The  ERA  program  in  Corpus  Christi  produced  gains,  relative  to  the  control 
group,  in  several  measures  of  employment  retention  and  earnings  in  the  cu- 
mulative four-year  follow-up  period.  In  addition,  there  are  some  indications 
that  ERA  may  have  led  to  increased  advancement,  compared  with  the  control 
group  level. 
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• ERA  decreased  the  amount  of  food  stamp  assistance  received  but  had  no  ef- 
fect on  the  amount  of  TANF  assistance  received.  ERA  increased  average  an- 
nual income  over  the  cumulative  follow-up  period. 

Control  Group  Outcomes 

Control  group  outcomes  during  the  four-year  follow-up  period  in  Texas  represent  what 
would  have  happened  in  the  absence  of  the  ERA  program  and  what  the  ERA  program  was 
trying  to  improve  on.  Average  earnings  for  control  group  members  in  Corpus  Christi  were  the 
lowest  among  all  the  ERA  control  groups  examined  in  this  report,  reflecting  a high  level  of 
disadvantage  in  this  sample.  Average  quarterly  employment  was  under  50  percent,  and  average 
earnings  were  under  $4,500  per  year  for  control  group  members  over  the  four-year  follow-up 
period  (Table  3.3).  Figure  3.1  shows  that  the  control  group’s  average  earnings  increased  steadily 
(although  modestly)  over  the  follow-up  period,  without  a corresponding  increase  in  the  em- 
ployment rate.5  This  may  indicate  that  there  was  some  advancement  over  time  in  the  control 
group,  although  the  data  cannot  indicate  whether  the  earnings  increase  is  due  to  more  hours  of 
work  or  higher  wages. 6 While  most  of  the  control  group  received  at  least  one  TANF  payment  in 
the  first  year  of  the  follow-up  period,  in  the  fourth  year  of  follow-up,  only  1 8 percent  of  the 
control  group  received  at  least  one  TANF  payment  (Appendix  Table  A.2). 

Employment  and  Employment  Retention 

Given  the  stipend  and  postemployment  services,  it  was  expected  that  the  ERA  program 
in  Corpus  Christi  would  have  effects  on  employment  retention.  The  Corpus  Christi  program  did 
generate  gains  in  several  measures  of  employment  retention  in  the  cumulative  four-year  follow- 
up period  (Table  3.3). 7 ERA  increased  average  quarterly  employment  by  3.8  percentage  points, 
or  by  about  8 percent  of  the  control  group  level.  ERA  also  increased  the  percentage  of  program 
group  members  who  had  an  employment  spell  of  at  least  four  quarters,  by  6.5  percentage  points 
— almost  12  percent  over  the  control  group  level.  In  addition,  ERA  increased  the  average 

5People  who  are  not  employed  in  a given  quarter  have  earnings  of  zero,  and  these  zeros  are  included 
in  average  earnings.  Thus,  average  earnings  could  increase  over  time  if  employment  levels  increased  as 
well  as  if  the  earnings  of  those  who  were  working  increased. 

6Many  studies  show  a similar  pattern  in  which  TANF  applicants’  and  new  recipients’  earnings  increase 
over  time  as  they  enter  into  the  labor  market,  not  necessarily  as  the  result  of  any  program. 

7Texas  program  impacts  do  not  appear  to  be  diluted  by  sample  members  who  never  qualified  for  TANF 
and  were  therefore  never  eligible  for  ERA  services.  This  measure  is  considered  nonexperimental  because  it 
includes  only  the  subset  of  the  full  research  sample  who  received  TANF  in  a certain  time  period  after  random 
assignment.  Nonexperimental  impacts  for  sample  members  who  received  TANF  in  the  first  or  second  quarter 
of  random  assignment  were  compared  with  impacts  for  those  who  did  not.  There  are  no  statistically  significant 
differences  in  the  impacts  between  these  groups,  which  suggests  that  the  dilution  did  not  have  an  effect  on  the 
results.  This  is  the  case  for  Corpus  Christi  and  Fort  Worth  (the  two  Texas  sites  that  experienced  positive 
economic  impacts). 
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Figure  3.1 

Impacts  on  Employment  and  Earnings  Over  Time: 
Corpus  Christi 
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SOURCE:  MDRC  calculations  from  administrative  records  for  the  State  of  Texas. 

NOTES:  "RA"  refers  to  the  quarter  of  random  assignment. 

Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings. 
Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  and  * = 10 
percent. 
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length  of  the  longest  employment  spell  by  0.4  quarter  — almost  8 percent  over  the  control 
group  average. 

The  ERA  program  in  Corpus  Christi  had  its  most  pronounced  effect  on  employment  re- 
tention by  helping  program  group  members  who  would  have  worked  for  a short  period  of  time 
retain  employment  for  a longer  period.  ERA  decreased  the  percentage  of  program  group 
members  who  worked  less  than  25  percent  of  the  quarters  in  the  follow-up  period  by  almost  30 
percent,  relative  to  the  control  group,  and  similarly  decreased  the  proportion  of  program  group 
members  whose  longest  employment  spell  was  only  one  or  two  quarters  (Appendix  Table  A.3). 
The  effects  of  the  ERA  program  in  Corpus  Christi  on  employment  retention  also  stemmed 
partially  from  helping  program  group  members  find  new  jobs  more  quickly  after  the  loss  of  a 
job.  ERA  decreased  the  average  length  of  the  longest  unemployment  spell  by  about  0.4  quarter 
(Table  3.3),  and  at  least  part  of  this  effect  was  due  to  the  program’s  helping  individuals  find  a 
job  more  quickly  after  loss  of  employment. 

As  can  be  seen  in  Figure  3.1,  the  ERA  program  in  Corpus  Christi  did  not  produce  statis- 
tically significant  impacts  on  employment  until  the  last  quarter  of  Year  2,  and  it  had  its  largest 
effect  on  employment  in  the  last  quarter  of  Year  3,  when  it  increased  employment  by  almost  7 
percentage  points  — a 14  percent  increase  above  the  control  group  level.  ERA  continued  to 
increase  quarterly  employment  in  Corpus  Christi  through  the  fourth  year  of  follow-up. 

Earnings  and  Advancement 

The  ERA  program  in  Corpus  Christi  increased  earnings  and  may  have  led  to  job  ad- 
vancement, compared  with  the  control  group  level.  ERA  first  generated  earnings  gains,  relative 
to  the  control  group,  in  the  first  year  of  follow-up.  After  these  early  gains,  there  were  no 
statistically  significant  impacts  on  earnings  for  a few  quarters,  until  they  appeared  again  in  the 
last  quarter  of  Year  2.  In  the  last  two  years  of  the  follow-up  period,  ERA  increased  average 
earnings  by  $175  to  $260  per  quarter,  an  increase  of  14  percent  to  21  percent  over  the  control 
group’s  quarterly  earnings.  It  would  be  expected  that  ERA  would  lead  to  increased  earnings  for 
program  group  members,  relative  to  control  group  members,  due  to  its  effects  on  employment 
retention.  However,  ERA  had  a larger  effect  on  earnings  than  can  be  explained  just  by  its  effect 
of  increasing  the  number  of  quarters  employed.  In  the  last  quarter  of  follow-up,  ERA  increased 
earnings  by  about  $250,  an  increase  of  20  percent  above  the  control  group  earnings  level.  In 
addition,  ERA  produced  an  employment  increase  of  almost  5 percentage  points  in  that  quarter, 
or  about  10  percent  above  the  control  group  level.  Because  the  earnings  impact  is  larger  in 
percentage  terms  than  the  employment  impact,  this  suggests  that  ERA  may  have  led  to  ad- 
vancement gains,  such  as  increases  in  hours  or  weeks  worked  or  in  wage  rates,  relative  to  the 
control  group. 
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The  ERA  program  in  Corpus  Christi  produced  gains  in  earnings  through  the  end  of  the 
four-year  follow-up  period,  even  after  program  group  members  would  have  lost  eligibility  for 
the  stipends  and  services.  In  fact,  the  ERA  program  in  Corpus  Christi  had  its  largest  effect  on 
earnings  in  the  fourth  year  of  follow-up,  suggesting  the  possibility  that  the  program  may  lead  to 
even  longer-tenn  gains  (Appendix  Table  A.2). 

Over  the  cumulative  four-year  follow-up  period,  the  ERA  program  in  Corpus  Christi 
generated  gains  in  several  measures  of  earnings  and  advancement  for  the  program  group, 
compared  with  the  control  group.  ERA  increased  average  annual  earnings  by  $640,  or  by 
almost  15  percent  of  control  group  earnings  (Table  3.3).  ERA  also  increased  the  percentage  of 
quarters  in  which  program  group  members  were  earning  more  than  $3,500,  by  about  2 percent- 
age points  — an  18  percent  increase  relative  to  the  control  group  level.  In  addition,  a nonexpe- 
rimental  measure  of  average  earnings  per  quarter  employed  is  higher  for  program  group 
members  than  for  control  group  members,  which  provides  evidence  that  there  was  some 
advancement  for  program  group  members  relative  to  control  group  members  (Appendix  Table 
A.3).8  ERA  did  not  have  a statistically  significant  impact,  however,  on  one  of  the  main  meas- 
ures of  advancement:  the  percentage  of  people  who  made  at  least  $250  more  in  their  highest- 
earning  quarter  in  Year  4 than  in  their  highest-earning  quarter  in  Year  1. 9 

Looking  at  a broader  definition  of  advancement,  the  ERA  program  in  Corpus  Christi 
did  produce  gains  in  some  measures  of  the  types  of  skills  required  at  jobs  and  in  some  measures 
of  benefits  at  the  jobs  that  sample  members  held  at  the  time  of  the  12-month  survey  (Appendix 
Table  A.4).  For  example,  ERA  increased  the  percentage  of  program  group  members  whose 
employers  provided  paid  vacation  days  by  close  to  8 percentage  points  — almost  doubling  the 
percentage  for  the  control  group. 

Public  Assistance  and  Average  Annual  Income 

The  ERA  program  in  Corpus  Christi  decreased  the  amount  of  food  stamp  assistance  re- 
ceived, but  it  increased  average  annual  income  over  the  four-year  follow-up  period  (Table  3.3). 
(This  measure  of  income  does  not  include  the  value  of  the  stipend.)  Program  group  members 
received  $134  less  in  annual  food  stamp  benefits,  on  average,  than  the  control  group  received. 
ERA  did  not  affect  the  amount  of  TANF  assistance  received  by  program  group  members. 

8This  measure  is  considered  nonexperimental  because  it  includes  only  a subset  of  the  full  report  sample. 
Because  people  who  are  employed  in  the  program  group  may  have  different  characteristics  than  people  who 
are  employed  in  the  control  group,  differences  in  these  outcomes  may  not  be  attributable  to  the  ERA  program. 

9This  measure  has  a couple  of  important  limitations.  First,  it  does  not  measure  advancement  that  may  have 
occurred  by  moves  from  unemployment  in  Y ear  1 into  employment.  If  ERA  was  able  to  help  people  move 
from  unemployment  into  better  jobs  than  similar  people  in  the  control  group,  then  this  would  be  a form  of 
advancement  as  well.  Second,  there  is  evidence  that  the  program  may  have  led  to  advancement  for  program 
group  members  within  the  first  year  of  follow-up;  this  measure  would  not  capture  that  type  of  advancement. 
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Largely  due  to  the  program’s  effects  on  earnings,  ERA  increased  annual  income  by  an  average 
of  $475,  a 6 percent  increase  over  the  control  group  level.  Including  the  value  of  stipends 
received,  ERA  increased  average  annual  income  by  $604  over  the  cumulative  four-year  follow- 
up period,  compared  with  the  control  group  level  (not  shown). 10 

Fort  Worth  ERA  Site:  Economic  Impacts 

Economic  impacts  of  the  Fort  Worth  ERA  program  can  be  summarized  as  follows: 

• The  ERA  program  in  Fort  Worth  generated  increases  in  some  measures  of 
employment,  employment  retention,  and  earnings  in  the  four-year  follow-up 
period,  relative  to  the  control  group  levels,  but  the  effects  were  concentrated 
in  the  second  and  third  years  of  follow-up.  There  is  also  evidence  that  the 
ERA  program  produced  some  gains  in  advancement,  compared  with  the  con- 
trol group  level,  in  the  four-year  follow-up  period. 

• ERA  increased  average  annual  income  and  the  amount  of  food  stamps  re- 
ceived in  the  cumulative  four-year  follow-up  period. 

Control  Group  Outcomes 

As  would  be  expected,  employment  and  earnings  were  lower  for  control  group  mem- 
bers in  Fort  Worth  than  for  control  group  members  in  other  target  groups  in  this  report  who 
were  already  employed  at  study  entry  (presented  in  Chapters  4 and  5).  However,  compared  with 
the  control  group  outcomes  for  other  samples  in  the  programs  presented  in  this  chapter,  the  Fort 
Worth  outcomes  are  not  remarkable.  The  average  quarterly  employment  rate  was  under  50 
percent,  and  average  earnings  were  just  under  $5,300  per  year  for  control  group  members  over 
the  four-year  follow-up  period  (Table  3.4).  Figure  3.2  shows  that  average  earnings  increased 
steadily  over  the  follow-up  period  — without  a corresponding  increase  in  employment  — 
which  indicates  there  was  some  advancement  over  time  in  the  control  group.  Table  3.6  shows 
that  while  most  of  the  control  group  received  at  least  one  TANF  payment  in  the  first  follow-up 
year,  only  24  percent  of  the  control  group  received  at  least  one  TANF  payment  in  the  fourth 
year. 


Employment  and  Employment  Retention 

The  ERA  program  in  Fort  Worth  did  not  produce  statistically  significant  impacts  on  any 
of  the  main  measures  of  employment  or  employment  retention  over  a cumulative  four-year 


10People  who  entered  the  study  late  in  the  follow-up  period  in  all  three  Texas  sites  had  approximately  21 
months  to  access  the  stipend  before  the  program  ended  (Martinson  and  Hendra,  2006). 
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Earnings  ($)  Percentage  employed 


The  Employment  Retention  and  Advancement  Project 


Figure  3.2 

Impacts  on  Employment  and  Earnings  Over  Time: 
Fort  Worth 


* 


Quarter  after  random  assignment 

SOURCE:  MDRC  calculations  from  administrative  records  for  the  State  of  Texas. 

NOTES:  "RA"  refers  to  the  quarter  of  random  assignment. 

Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings. 
Statistical  significance  levels  are  indicated  as  follows:  ***  = lpercent;  **  = 5 percent;  and  * = 
10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Table  3.6 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income, 

by  Year  of  Follow-Up: 

Fort  Worth 

ERA  Control  Difference  Standard 

Outcome  Group  Group  (Impact)  P-Value  Error 

Follow-up  Year  1 


Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 

Follow-up  Year  2 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 

Follow-up  Year  3 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 

Follow-up  Year  4 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 


68.4 

68.5 

0.0 

0.984 

2.2 

47.2 

47.2 

0.0 

0.987 

1.8 

23.6 

25.5 

-1.9 

0.357 

2.0 

14.6 

15.1 

-0.5 

0.761 

1.7 

4,235 

4,289 

-54 

0.851 

286 

80.2 

79.9 

0.2 

0.901 

1.9 

92.3 

91.3 

0.9 

0.487 

1.4 

8,173 

8,068 

105 

0.695 

269 

69.6 

63.2 

6.4  *** 

0.003 

2.2 

49.1 

45.7 

3.4  * 

0.070 

1.9 

29.0 

27.0 

1.9 

0.362 

2.1 

21.7 

19.4 

2.2 

0.244 

1.9 

5,477 

5,010 

466 

0.184 

351 

49.1 

48.7 

0.4 

0.883 

2.4 

79.8 

80.4 

-0.7 

0.736 

1.9 

8,727 

8,089 

638  * 

0.054 

331 

65.8 

64.2 

1.6 

0.471 

2.2 

49.6 

46.4 

3.2 

0.100 

2.0 

33.8 

28.7 

g j ** 

0.022 

2.2 

24.7 

21.7 

3.0 

0.136 

2.0 

6,395 

5,482 

914  ** 

0.022 

399 

32.0 

35.4 

-3.3 

0.141 

2.3 

75.7 

75.9 

-0.2 

0.915 

2.0 

9,416  8,379  1,036  ***  0.006  380 


63.8 

62.2 

1.6 

0.475 

2.2 

48.0 

48.2 

-0.1 

0.945 

2.0 

32.2 

33.7 

-1.4 

0.529 

2.3 

26.1 

25.0 

1.1 

0.599 

2.1 

6,715 

6,245 

470 

0.275 

431 

21.8 

23.5 

-1.7 

0.410 

2.0 

70.2 

72.1 

-1.9 

0.374 

2.1 

9,691 

9,086 

605 

0.143 

412 

Sample  size  (total  = 1,572)  784  788 

(continued) 
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Table  3.6  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  represents  the  sum  of  the  Ul-reported  earnings,  TANF,  and  food  stamps. 


follow-up  period  (Table  3.4).  However,  the  program  did  generate  increases,  relative  to  the 
control  group,  in  some  measures  of  employment  and  employment  retention  in  the  second  and 
third  years  of  follow-up  (Table  3.6). 

The  gains  in  employment  in  some  of  the  quarters  in  the  second  and  third  years  of  fol- 
low-up (Figure  3.2)  were  large  enough  to  generate  an  increase  in  average  quarterly  employment 
in  Year  2 (Table  3.6).  ERA  also  increased  the  percentage  employed  in  every  quarter  of  Year  3 
by  5 percentage  points,  or  by  almost  18  percent  of  the  control  group  level.  These  effects 
disappeared  by  the  fourth  year  of  follow-up,  however. 

Given  the  stipend  and  postemployment  services,  it  was  expected  that  the  program’s  ef- 
fects would  be  on  employment  retention.  While  the  program  in  Fort  Worth  did  increase  em- 
ployment retention  in  the  middle  of  the  follow-up  period,  the  effects  were  not  as  persistent  as 
those  for  the  program  in  Corpus  Christi. 

Earnings  and  Advancement 

The  ERA  program  in  Fort  Worth  produced  some  earnings  gains  and  may  have  led  to  ad- 
ditional advancement,  compared  with  the  control  group  levels.  ERA  increased  quarterly  earnings, 
relative  to  the  control  group,  at  the  end  of  Year  2 and  throughout  most  of  Year  3.  It  would  be 
expected  that  ERA  would  lead  to  increased  earnings  for  program  group  members,  relative  to 
control  group  members,  due  to  its  effects  on  employment  retention.  However,  Fort  Worth  had  a 
larger  and  more  persistent  effect  on  earnings  than  could  be  explained  by  its  effect  on  increasing 
the  number  of  quarters  employed.  In  the  quarters  in  which  there  were  statistically  significant 
impacts  on  earnings,  the  impacts  ranged  from  $200  to  $315  per  quarter,  an  increase  of  16  percent 
to  24  percent  over  the  control  group’s  average  earnings.  In  the  same  quarters,  program  group 
employment  rates  were  5 percent  to  1 1 percent  higher  than  the  control  group  employment  rates. 
Because  the  earnings  impacts  are  larger  in  percentage  tenns  than  the  employment  impacts,  this 
suggests  that  ERA  may  have  led  to  advancement,  such  as  increases  in  hours  or  weeks  worked  or 
wage  increases. 


72 


The  ERA  program  in  Fort  Worth  did  not  produce  an  impact  on  earnings  in  the  cumula- 
tive four-year  follow-up  period,  relative  to  the  control  group  (Table  3.4).  However,  the  nonex- 
perimental  measure  of  average  earnings  per  quarter  employed  — described  above,  in  the 
discussion  of  Corpus  Christi  — is  higher  for  program  group  members  than  for  control  group 
members  (Appendix  Table  A.6).  This  also  indicates  that  there  was  some  advancement  gain  for 
program  group  members. 

The  Fort  Worth  program  generated  statistically  significant  impacts  on  some  measures 
of  the  types  of  skills  required  at  jobs  that  sample  members  held  at  the  time  of  the  12-month 
survey  (Appendix  Table  A.7).  ERA  had  a particularly  large  effect  on  the  percentage  of  program 
group  members  who  worked  with  computers  at  least  monthly  in  their  job;  almost  three  times  as 
many  of  them  reported  this  activity,  compared  with  the  control  group. 

Public  Assistance  and  Average  Annual  Income 

The  ERA  program  in  Fort  Worth  produced  an  average  annual  increase  in  food  stamps 
of  about  $160,  compared  with  the  level  seen  in  the  control  group  over  the  cumulative  four-year 
follow-up  period  (Table  3.4).  ERA  also  increased  average  annual  income  by  about  $600,  or  by 
7 percent  of  the  control  group  level.  (This  measure  of  income  does  not  include  the  value  of  the 
stipend.)  Including  the  value  of  stipends  received,  ERA  increased  average  annual  income  by 
almost  $700  over  the  cumulative  four-year  follow-up  period,  compared  with  the  control  group 
level  (not  shown).11  The  ERA  program  in  Fort  Worth  had  no  effect  on  TANF  benefits  during 
the  four-year  follow-up  period. 

For  reasons  that  are  unclear,  the  Fort  Worth  program  increased  the  amount  of  food 
stamp  assistance  received  in  each  of  the  first  three  years  of  follow-up  (not  shown).  Due  to  the 
effects  on  earnings  and  food  stamp  assistance,  the  ERA  program  in  Fort  Worth  increased 
income  in  the  second  and  third  years  of  follow-up.  In  Year  3,  the  income  of  program  group 
members  averaged  over  $1,000  (12  percent)  more  than  the  income  for  control  group  members 
(Table  3.6). 

Houston  ERA  Site:  Economic  Impacts 

Economic  impacts  of  the  Houston  ERA  program  can  be  summarized  as  follows: 

• The  ERA  program  in  Houston  did  not  produce  any  changes,  relative  to  the 
control  group,  in  employment,  employment  retention,  or  earnings  in  the  four- 
year  follow-up  period.  There  is  also  no  evidence  that  ERA  led  to  advance- 
ment gains,  compared  with  the  level  of  advancement  for  the  control  group. 


"Martinson  and  Hendra  (2006). 
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• ERA  increased  the  amount  of  TANF  received  throughout  the  follow-up  pe- 
riod but  did  not  have  any  effect  on  food  stamp  receipt  or  average  annual  in- 
come. 

Control  Group  Outcomes 

Average  quarterly  employment  and  average  earnings  were  quite  low  for  control  group 
members  in  Houston,  even  compared  with  those  in  other  ERA  programs  targeting  similar 
individuals  (not  employed  at  study  entry).  Table  3.5  shows  that  average  quarterly  employment 
was  less  than  45  percent  and  that  average  earnings  were  under  $5,000  per  year  for  control  group 
members  over  the  four-year  follow-up  period.  Average  earnings  increased  modestly  over  the 
follow-up  period,  without  a corresponding  increase  in  employment,  which  indicates  that  there 
was  some  advancement  over  time  for  those  in  the  control  group  (Appendix  Table  A.9).  While 
most  of  the  control  group  received  at  least  one  TANF  payment  in  the  first  follow-up  year,  only 
about  30  percent  of  the  control  group  received  at  least  one  TANF  payment  in  the  fourth  year 
(Appendix  Table  A.9). 

Employment  and  Employment  Retention 

The  Houston  ERA  program  did  not  generate  any  changes,  relative  to  the  control  group, 
in  the  main  measures  of  employment  or  employment  retention  in  the  four-year  follow-up  period 
(Table  3.5).  Corpus  Christi  and  Fort  Worth  both  had  impacts  on  employment  retention,  and  it 
took  time  to  produce  these  effects.  In  Houston,  no  effects  on  employment  or  employment 
retention  emerged  over  time. 

Patterns  of  stipend  receipt  were  different  in  Houston  than  in  Corpus  Christi  and  Fort 
Worth.  While  about  20  percent  of  program  group  members  in  Houston  ever  received  a stipend 
(similar  to  the  rate  in  Fort  Worth),  a much  lower  percentage  of  program  group  members  in 
Houston  received  a significant  number  of  stipends.  Only  9 percent  of  program  group  members 
in  Houston  who  ever  received  a stipend  received  1 1 or  more  stipends,  compared  with  close  to 
50  percent  in  Corpus  Christi  and  Fort  Worth.  This  result  is  probably  attributable  to  the  weak 
implementation  of  the  ERA  program  in  Houston,  where  staff  experienced  significant  difficulties 
putting  postemployment  services  in  place  (including  the  stipend). 

Earnings  and  Advancement 

The  ERA  program  in  Houston  did  not  produce  any  increases  in  any  measures  of  earn- 
ings or  advancement  compared  with  the  control  group  levels  (Table  3.5). 
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Public  Assistance  and  Average  Annual  Income 

On  average,  ERA  raised  annual  TANF  benefits  in  Houston  by  about  $60  for  the  pro- 
gram group,  compared  with  the  control  group  level,  over  the  cumulative  four-year  follow-up 
period  (Table  3.5).  But  the  increase  was  not  large  enough  to  produce  statistically  significant 
differences  in  income,  not  including  the  value  of  stipends  received.  Even  including  the  value  of 
stipends,  ERA  did  not  increase  average  annual  income  over  the  cumulative  four-year  follow-up 
period,  compared  with  the  control  group  level.  The  program  did  not  produce  any  changes  in  the 
amounts  of  food  stamp  assistance  received. 

Texas  ERA  Program:  Pattern  of  Findings 

While  the  Corpus  Christi  ERA  program  generated  some  increases  in  earnings  in  the 
first  year  of  follow-up,  it  was  not  until  the  end  of  Year  2 that  the  Corpus  Christi  and  Fort  Worth 
programs  started  producing  their  largest  and  most  sustained  effects  on  employment  retention 
and  earnings. 12  This  delay  may  have  been  the  result  of  the  way  the  ERA  and  control  services 
were  structured.  Program  and  control  group  members  were  not  employed  at  the  time  they 
entered  the  study,  and,  as  discussed  above,  preemployment  services  ended  up  being  very  similar 
for  the  ERA  and  control  groups.  After  finding  employment,  both  program  and  control  group 
members  had  four  months  in  which  their  earnings  were  disregarded  in  the  calculation  of  their 
TANF  grant,  which  essentially  gave  them  an  employment  “stipend”  of  sorts  (as  they  generally 
received  almost  their  full  TANF  grant  in  addition  to  their  earnings  during  that  time).  Thus,  the 
ERA  and  control  groups  received  similar  services  through  the  end  of  the  earnings  disregard 
period:  both  groups  received  preemployment  services,  minimal  postemployment  services,  and  a 
stipend  in  the  form  of  the  earnings  disregard.  Only  after  the  disregard  period  was  there  are  a 
large  difference  in  program  services. 

Economic  Impact  Findings,  by  Study  Cohort 

Despite  the  fact  that  program  implementation  and  the  marketing  of  the  stipend  appear  to 
have  improved  over  time  in  both  the  Corpus  Christi  and  the  Fort  Worth  ERA  program,  there  are 
no  statistically  significant  differences  in  impacts  when  analyzed  by  study  cohort. 


12The  interim  report  on  the  Texas  program  (cited  in  Appendix  E)  presents  analysis  of  two  years  of  follow- 
up data  for  a subsample  of  sample  members  who  had  a full  two  years  of  follow-up  data  at  the  time.  The  results 
presented  in  that  report  reflect  modest  and  scattered  effects;  however,  additional  follow-up  has  changed  the 
story. 
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Texas  ERA  Program:  Summary  and  Conclusions 

The  Texas  ERA  program  was  designed  to  provide  both  job  search  assistance  and  post- 
employment services  (which  could  include  employer  site  visits  and  reemployment  assistance)  to 
individuals  applying  for  or  receiving  cash  assistance,  most  of  whom  were  not  working  when 
they  entered  the  program.  To  encourage  employment  retention  and  advancement,  the  program 
offered  a monthly  stipend  of  $200  for  TANF  leavers  working  at  least  30  hours  a week  after 
receiving  an  earned  income  disregard  for  four  months.  The  control  group  received  preemploy- 
ment services  that  focused  on  quick  job  entry,  followed  by  limited  postemployment  services. 

Corpus  Christi  ERA  Site 

The  ERA  program  was  strongly  implemented  in  Corpus  Christi,  with  key  features  put 
in  place  relatively  quickly,  including  a developed  strategy  for  marketing  the  stipend  and  a strong 
postemployment  service  component  featuring  site  visits  to  employers  to  address  job-related 
issues  and  job  advancement.  While  postemployment  services  were  different  for  the  program 
and  control  groups,  services  in  the  preemployment  phase  and  through  the  earnings  disregard 
period  ended  up  being  similar  for  the  two  groups. 

Most  program  group  members  in  Corpus  Christi  had  contact  with  ERA  staff  or  staff 
from  an  employment  program,  and  one-quarter  had  contact  within  four  weeks  prior  to  the  12- 
month  survey.  About  two-thirds  of  program  group  members  participated  in  a job  search 
activity.  While  participation  in  postemployment  services  was  not  as  pervasive  — reflecting  the 
voluntary  nature  of  those  services  — over  one-quarter  of  program  group  members  received  help 
with  retention  or  advancement  in  the  first  12  months  after  study  entry.  Among  the  three  Texas 
ERA  sites,  stipend  receipt  rates  were  highest  in  Corpus  Christi,  where  30  percent  of  all  program 
group  members  received  a stipend. 

Compared  with  the  control  group,  the  ERA  program  in  Corpus  Christi  increased  the 
level  of  contact  with  ERA  staff  or  staff  from  an  employment  program  and  the  proportion  of 
clients  who  received  assistance  with  employment  retention  and  advancement  issues,  particularly 
when  they  were  working.  Reflecting  the  similarities  in  preemployment  services  across  the  two 
groups,  ERA  did  not  raise  the  level  of  participation  in  job  search  activities.  But  ERA  doubled 
the  percentage  of  program  group  members  who  received  help  with  retention  or  advancement 
relative  to  the  control  group. 

The  ERA  program  in  Corpus  Christi  produced  increases  in  several  measures  of  em- 
ployment retention,  earnings,  and  advancement  relative  to  the  control  group  in  the  four-year 
follow-up  period.  Moreover,  the  program  was  producing  gains  in  employment  retention  and 
earnings  through  the  end  of  the  four-year  follow-up  period,  even  after  program  group  members 
would  have  lost  eligibility  for  the  stipends  and  services.  Indeed,  the  ERA  program  in  Corpus 
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Christi  had  its  largest  effect  on  earnings  in  the  fourth  year  of  follow-up,  suggesting  that  the 
program  may  lead  to  even  longer-term  gains.  The  program  in  Corpus  Christi  also  increased 
average  annual  income  over  the  cumulative  four-year  follow-up  period. 

Fort  Worth  ERA  Site 

The  ERA  program  in  Fort  Worth  experienced  some  operational  difficulties  early  in  the 
study  period,  particularly  in  designing  preemployment  services  and  managing  the  client  flow 
through  program  activities.  Initially,  the  program  had  a strong  emphasis  on  assessment  and  on 
removing  employment  barriers,  which  sometimes  delayed  program  group  members’  movement 
into  employment  and  postemployment  services.  The  ERA  program  in  Fort  Worth  made 
significant  improvements  over  time,  however,  including  adding  more  structured  job  search 
services  and  stronger  postemployment  services  that  included  regular  employer  site  visits. 

Despite  the  early  implementation  problems  in  the  Fort  Worth  program,  levels  of  contact 
with  ERA  staff  or  staff  from  an  employment  program  were  similar  among  Fort  Worth  and 
Corpus  Christi  program  group  members.  Reported  levels  of  both  participation  in  job  search 
activities  and  receipt  of  retention  and  advancement  assistance  were  even  higher  among  program 
group  members  in  Fort  Worth  than  among  those  in  Corpus  Christi.  Stipend  receipt  rates  in  Fort 
Worth  were  lower  than  in  Corpus  Christi,  with  about  20  percent  of  all  program  group  members 
ever  receiving  a stipend. 

As  was  the  case  in  Corpus  Christi,  the  ERA  program  in  Fort  Worth  increased  the  pro- 
portion of  program  group  members  who  had  contact  with  ERA  staff  or  staff  from  an  employ- 
ment program  as  well  as  the  proportion  who  received  assistance  with  job  advancement  services 
— compared  with  the  control  group.  Reflecting  the  similarities  in  preemployment  services 
across  the  program  and  control  groups,  ERA  did  not  raise  the  level  of  participation  in  job  search 
activities.  The  ERA  program  in  Fort  Worth  substantially  increased  the  percentage  of  program 
group  members  who  received  help  with  retention  or  advancement  relative  to  the  control  group. 

The  ERA  program  in  Fort  Worth  increased  employment  retention,  earnings,  and  ad- 
vancement, particularly  in  the  middle  of  the  four-year  follow-up  period,  but  the  effects  were  not 
as  persistent  as  those  for  the  program  in  Corpus  Christi.  The  program  in  Fort  Worth  also 
increased  average  annual  income  over  the  cumulative  four-year  follow-up  period. 

Several  factors  may  account  for  why  the  ERA  program  in  Fort  Worth  had  less  persis- 
tent impacts  on  employment  retention  than  the  program  in  Corpus  Christi.  First,  implementation 
of  the  program  was  less  strong,  and  while  it  improved  over  time,  this  may  have  led  to  the  later 
emergence  and  smaller  scale  of  impacts  in  Fort  Worth.  (Many  program  group  members  went 
through  the  program  before  implementation  had  improved.)  Second,  the  stipend  take-up  rate 
was  lower  in  Fort  Worth  than  in  Corpus  Christi:  22  percent  of  program  group  members  ever 


77 


received  a stipend  in  Fort  Worth,  compared  with  30  percent  of  program  group  members  in 
Corpus  Christi.  In  addition,  stipend  receipt  among  those  estimated  to  have  met  the  eligibility 
criteria  to  receive  it  was  lower  in  Fort  Worth  than  in  Corpus  Christi. 13 

Houston  ERA  Site 

Of  the  three  Texas  ERA  sites,  Houston  moved  the  most  slowly  to  get  ERA’s  retention 
and  advancement  services  off  the  ground.  Furthermore,  some  components  of  the  model  were 
never  implemented.  For  much  of  the  study  period,  this  program  placed  a high  priority  on 
providing  preemployment  services,  rather  than  on  developing  postemployment  services.  While 
some  postemployment  services,  such  as  career  assessment,  were  eventually  put  in  place,  even 
then  the  services  did  not  include  some  key  elements,  such  as  employer  site  visits,  that  were  used 
in  the  other  two  Texas  sites. 

The  program  group  members  in  Houston  had  similar  levels  of  ever  having  contact  with 
ERA  staff  or  staff  from  an  employment  program  as  the  program  groups  in  Corpus  Christi  and 
Fort  Worth,  but  the  percentage  having  contact  within  four  weeks  prior  to  the  survey  was  lower 
in  Houston  than  in  the  other  sites.  While  the  level  of  participation  in  job  search  was  high  among 
program  group  members  in  Houston  — reflecting  the  site’s  emphasis  on  preemployment 
services  — a smaller  percentage  of  program  group  members  reported  that  they  had  received 
help  with  retention  or  advancement  in  the  first  12  months  after  study  entry,  compared  with 
program  group  members  in  the  other  sites.  The  percentage  of  program  group  members  in 
Houston  who  ever  received  a stipend  was  approximately  the  same  as  in  Fort  Worth,  but  a much 
lower  percentage  received  a significant  number  of  stipends.  Unlike  the  other  Texas  sites,  the 
ERA  program  in  Houston  did  not  increase  the  proportion  of  program  group  members  who 
reported  having  contact  with  ERA  staff  or  staff  from  an  employment  program  or  receiving  help 
with  retention  or  advancement,  relative  to  control  group  members.  The  ERA  program  in 
Houston  did  not  have  an  effect  on  employment,  employment  retention,  earnings,  or  advance- 
ment. 


Cross-Site  Conclusions 

The  results  from  the  Texas  ERA  program  reinforce  a growing  body  of  evidence  show- 
ing that  combining  well-implemented  services  and  financial  incentives  conditioned  on  em- 
ployment can  lead  to  positive  employment  and  earnings  impacts.14  Corpus  Christi  had  the 
strongest  implementation  of  the  Texas  ERA  program  and  the  highest  stipend  receipt  rate,  and  it 
had  the  largest  and  most  persistent  impacts  on  economic  outcomes.  In  Fort  Worth,  initial 


13Martinson  and  Hendra  (2006). 
14Michalopoulos  (2005). 
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implementation  difficulties  and  the  lower  stipend  receipt  rate  may  have  led  this  program  to  have 
less  consistent  economic  impacts  than  those  in  Corpus  Christi.  Implementation  of  the  program 
improved  over  time  in  Fort  Worth,  which  may  account  for  the  later  emergence  and  smaller  scale 
of  impacts.  In  Houston,  postemployment  services  and  stipend  take-up  were  weak  compared 
with  the  other  sites,  which  illustrates  the  importance  of  strong  program  implementation  as  well 
as  design. 

The  economic  impacts  may  have  been  diluted  in  the  Texas  ERA  program  to  some  ex- 
tent by  factors  related  to  the  research  design.  Program  administrators  wanted  to  enroll  sample 
members  into  the  Texas  ERA  program  when  they  were  not  yet  employed  because  it  was 
thought  that  it  might  be  helpful  to  encourage  people  to  think  about  retention  and  advancement 
as  they  were  looking  for  employment  and  to  build  relationships  with  staff  that  would  persist 
after  clients  found  jobs.  However,  both  the  preemployment  and  the  postemployment  services 
that  were  available  during  the  earnings  disregard  period  ended  up  being  fairly  similar  for  the 
program  and  control  groups.  The  more  intensive  ERA  postemployment  services  (such  as 
employer  site  visits)  did  not  generally  start  until  the  stipend  phase  of  the  program,  after  an 
individual  had  been  working  for  four  months.  Therefore,  the  impacts  may  have  been  diluted, 
since  some  sample  members  did  not  stay  in  the  program  past  the  earnings  disregard  period  and 
were  not  exposed  to  the  key  service  differential  between  the  ERA  and  control  groups. 15  This 
delay  in  service  differential  also  contributes  to  timing  of  the  impacts,  which  were  not  large  in 
Corpus  Christi  until  later  in  the  follow-up  period  and  did  not  appear  in  Fort  Worth  until  well 
into  the  follow-up  period. 

The  main  purpose  of  the  stipend  used  in  the  Texas  ERA  program  was  to  encourage  em- 
ployment retention.  Another  benefit,  however,  should  be  noted:  additional  income  was  provided 
to  these  low-income  families  through  the  stipend. 


^Expectations  about  future  stipends,  however,  might  have  affected  the  employment  behavior  of  program 
group  members  before  the  end  of  the  earnings  disregard  period. 
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Los  Angeles  Enhanced  Job  Club  (EJC)  ERA  Findings 


Los  Angeles  EJC  ERA  at  a Glance 

• Served  unemployed  TANF  recipients  who  were  required  to  search  for  employment 

• Offered  a job  club  that  sought  to  place  individuals  in  jobs  more  in  line  with  their  career 
interests 

• Emphasized  that  individuals  should  seek  jobs  in  their  area  of  career  interest  and  helped 
clients  seek  these  types  of  jobs 

• Was  compared  with  a control  group  that  was  required  to  participate  in  a traditional  j ob 
club,  emphasizing  seeking  any  type  of  j ob  that  could  result  in  a quick  placement 

• Based  on  field  research,  appeared  to  deliver  a different  message  than  the  traditional  job 
club  about  the  types  of  jobs  to  seek,  but  not  supported  by  survey  results  among  program 
group  members  a year  later 

• Resulted  in  no  improvements  in  employment  retention  or  advancement  outcomes 


Los  Angeles  EJC  ERA  Test:  Introduction 

The  goal  of  the  Los  Angeles  Enhanced  Job  Club  (EJC)  model  was  to  increase  employ- 
ment retention  and  advancement  among  unemployed  TANF  recipients  through  the  implementa- 
tion of  a distinct  job  club  model  that  had  a focus  on  career  development.  The  job  club  was  one 
component  of  the  larger  welfare-to-work  program  operated  in  Los  Angeles,  called  Greater 
Avenues  for  Independence  (GAIN)  — a program  that  was  strongly  focused  on  placing  welfare 
recipients  quickly  into  jobs.  The  study  enrolled  sample  members  from  June  2002  through 
December  2004. 

Origins  of  the  Test 

In  2000,  Los  Angeles  County  sought  to  increase  TANF  recipients’  employment  reten- 
tion as  well  as  their  long-tenn  self-sufficiency.  To  foster  these  goals,  the  county  developed  an 
“enhanced”  job  club  model  that,  in  addition  to  imparting  basic  job  search  skills,  featured  career 
planning  and  used  a “step-down”  approach  to  connect  clients  to  work.  The  step-down  approach 
involved  seeking  employment  within  a specific  occupational  field  of  interest  defined  by  the 
client.  The  EJC  model  sought  to  improve  on  Los  Angeles  County’s  previous  success  in  offering 
highly  effective  “traditional”  job  clubs  (TJC).  The  TJC  model  — which  used  a “work-first” 
approach  and  generally  placed  individuals  in  low-wage,  low-skill  jobs  — had  some  of  the 
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largest  employment  impacts  among  welfare-to-work  programs  studied,  when  compared  with  a 
control  group  that  was  not  mandated  to  participate  in  employment  services.16 

In  addition  to  the  TJC  model,  another  job  club  model  that  operated  in  Portland,  Oregon, 
as  part  of  the  National  Evaluation  of  Welfare-to-Work  Strategies  (NEWWS),  informed  the 
design  of  the  Los  Angeles  EJC  model.  The  Portland  model  was  also  built  on  the  hypothesis  that 
the  messages  delivered  in  job  clubs  about  what  type  of  job  a client  might  seek  could  make  a 
difference  in  eventual  economic  outcomes.  Unlike  the  EJC  model  studied  in  ERA,  however,  the 
Portland  model  emphasized  seeking  a job  that  had  a relatively  high  rate  of  pay,  fringe  benefits, 
and  an  opportunity  for  advancement.  The  EJC  model,  in  contrast,  emphasized  seeking  the  most 
desired  job  in  an  individual’s  area  of  interest,  which  might  not  necessarily  be  the  highest-paying 
job.  If  that  most  desired  job  was  not  attainable,  the  individual  could  “step  down”  to  the  next 
most  desired  job  in  that  field,  which  could  potentially  lead  to  the  most  desired  job. 

The  five- week  EJC  model  attempted  to  place  individuals  into  jobs  in  their  field  of  ca- 
reer interest  from  the  outset,  with  the  belief  that  this  approach  would  enable  them  to  keep  their 
jobs  longer,  engage  in  career  advancement  activities,  and  ultimately  move  into  better  jobs  along 
a career  path.  In  contrast,  the  TJC  model,  which  was  the  control  condition  in  this  test,  was  the 
conventional,  but  proven  effective,  three- week  job  club  workshop  that  the  county  had  been 
operating  since  1995.  The  TJC  model  aimed  to  get  clients  into  jobs  quickly,  regardless  of  the 
jobs’  field,  wage  rates,  or  promotional  opportunities.  It  was  reasoned  that  getting  people  quickly 
into  jobs  would  enable  them  to  obtain  earnings  faster,  develop  a longer  work  history  and 
positive  work  behavior  habits,  master  occupational  skills,  and  build  a social  network  that  could 
lead  to  better  job  opportunities. 

Labor  Market  Context 

The  Los  Angeles  EJC  ERA  test  began  just  as  the  area’s  labor  market  was  exiting  from  a 
downturn  caused  by  2001  recession.  Unemployment  rates  were  declining  from  the  high  reached 
during  the  recession  and  were  returning  to  prerecession  levels,  and  the  pace  of  the  recovery  in 
the  area’s  labor  market  was  faster  than  the  national  recovery.  Thus,  while  the  initial  conditions 
faced  by  sample  members  at  the  beginning  of  the  ERA  test  were  challenging,  the  labor  market 
was  expanding  for  almost  the  entirety  of  the  job  clubs’  operation  and  follow-up  period. 

Target  Population 

The  Los  Angeles  EJC  model  targeted  unemployed  TANF  recipients  who  were  man- 
dated to  participate  in  the  county’s  welfare-to-work  program  if  they  spoke  either  English  or 
Spanish  and  were  not  involved  in  a second  ERA  program  that  operated  in  Los  Angeles  County 

16Freedman,  Knab,  Gennetian,  and  Navarro  (2000). 
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(the  Reach  for  Success  [RFS]  program,  discussed  in  Chapter  4).  Table  3.2  shows  characteristics 
of  the  sample  at  the  time  the  study  began.  Over  40  percent  of  sample  members  had  been 
receiving  TANF  for  two  years  or  more  when  they  entered  the  study,  and  most  had  received 
TANF  cash  assistance  before.  Further,  only  27  percent  reported  employment  in  the  quarter  prior 
to  random  assignment  — the  lowest  among  the  sites  in  this  target  group.  The  duration  of  TANF 
receipt  and  the  low  prior  employment  rates  indicate  that  this  sample  was  disadvantaged. 

Significance  of  the  Test 

The  Los  Angeles  EJC  ERA  test  examined  whether  changes  in  the  design  of  a proven 
three-week  job  club,  administered  by  a welfare  agency,  could  improve  job  quality  and  increase 
employment  retention  and  advancement  for  unemployed  TANF  recipients.  Specifically,  this  site 
tested  whether  a mandatory  job  club  that  focused  on  career  development  activities  could  help 
individuals  find  jobs  that  they  could  retain  longer  and  be  more  likely  to  use  as  a basis  for 
advancement  than  the  county’s  more  traditional,  mandatory  job  club.  Chart  3.2  highlights 
important  differences  between  the  job  clubs  for  the  EJC  group  and  the  TJC  group  members. 

Los  Angeles  EJC  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

The  Los  Angeles  County  Department  of  Public  Social  Services  (DPSS)  and  the  Los 
Angeles  County  Office  of  Education  (LACOE)  jointly  operated  the  enhanced  and  traditional 
job  clubs.  These  job  clubs  were  part  of  California’s  mandatory  welfare-to-work  program  for 
unemployed  TANF  recipients,  known  as  the  Greater  Avenues  for  Independence  (GAIN) 
program. 

This  study  took  place  in  two  welfare  administration  regions  in  Los  Angeles  County. 
These  two  regions  included  the  San  Gabriel  Valley,  which  forms  the  eastern  suburbs  of  the  city 
of  Los  Angeles,  and  central  Los  Angeles,  which  encompasses  downtown  Los  Angeles  and  part 
of  the  neighborhood  of  South  Central  Los  Angeles. 

Staffing 

Job  club  facilitators  were  primarily  responsible  for  the  implementation  of  the  EJC  mod- 
el, even  though  several  other  types  of  staff  assisted.  In  fall  2002,  an  MDRC  contractor  trained 
these  facilitators  on  career  plan  development  using  the  step-down  approach.  Each  of  the  two 
regions  had  approximately  four  to  seven  EJC  facilitators  and  held  four  to  five  job  club  work- 
shops per  month,  on  average,  with  approximately  24  to  38  EJC  clients  attending  each  work- 
shop. The  TJC  facilitators  had  a similar  workshop  frequency  and  average  attendance. 
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The  Employment  Retention  and  Advancement  Project 
Chart  3.2 


Program-Control  Group  Differences: 
Los  Angeles  EJC  Test 


ERA  Group 

Control  Group 

Goals 

• Place  unemployed  TANF  recipients 
in  targeted  and  promotable  jobs  within 
identified  field  of  interest 

• Place  unemployed  TANF  recipients 
in  any  job  as  quickly  as  possible 

Staffing 

• Access  to  job  developer  and  job 
coach 

• Limited  access  to  job  developer  and 
no  access  to  job  coach 

Staff-client  engagement 

• Formal  process  for  identifying 
employment  barriers  and  assessing 
career  interests  and  feasibility  of  career 
goals 

• Career  plan  developed 

• No  formal  process  for  identifying 
employment  barriers  and  assessing 
career  interests  and  feasibility  of  career 
goals 

• No  career  plan  developed 

Job  preparation  and 
placement 

• Five-week  job  club  workshop 

• Place  unemployed  TANF  recipients 
in  targeted  and  promotable  jobs  within 
identified  field  of  interest  (“step-down” 
job  search  approach) 

• Three-week  job  club  workshop 

• Place  unemployed  TANF  recipients 
in  any  job  as  quickly  as  possible 
(“work-first”  job  search  approach) 

Retention  services 

• Job  placement  in  field  of  interest 

• Job  placement  in  any  job 

Advancement  services 

• Career  development  guidance 

• No  career  development  guidance 

Employer  linkages 

• Minimal  links  to  employers  through 
job  developer 

• No  links  to  employers 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


Other  LACOE  and  DPSS  staff  contributed  to  serving  EJC  clients,  although  the  model 
did  not  use  a team-based  staffing  structure,  as  was  the  case  in  some  other  ERA  models,  where 
multiple  staff  are  jointly  responsible  for  client  progress.  These  staff  included  job  coaches  who 
helped  program  group  members  research  potential  career  fields  and  job  developers  who 
matched  individuals  to  specific  job  openings  and  provided  one-on-one  assistance  with  resume 
preparation  and  the  use  of  career  development  software.  LACOE  and  DPSS  staff  were  also  the 
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TJC  model  facilitators.  Other  staff  that  assisted  the  EJC  facilitators,  however,  rarely  worked 
with  control  group  members. 

Individuals  in  both  the  EJC  and  the  TJC  group  maintained  their  TANF  case  manager, 
who  worked  with  them  on  issues  related  to  eligibility  and  supportive  services  that  were  normal- 
ly provided  through  the  TANF  program.  The  TANF  case  manager  did  not  distinguish  between 
or  provide  different  types  of  services  to  individuals  in  the  EJC  or  TJC  groups. 

The  staff  perfonnance  goals  for  the  EJC  and  TJC  facilitators  in  this  study  were  the 
same.  Staff  leading  both  types  of  job  clubs  was  expected  to  place  30  percent  of  their  workshop 
clients  in  jobs  (of  any  type)  by  the  end  of  the  job  club  session.  In  addition  to  the  same  perfor- 
mance goals,  there  was  little  to  no  distinction  in  staff  skills,  experience,  or  training  between  the 
EJC  and  TJC  facilitators. 

Staff-Client  Engagement 

Both  the  EJC  and  the  TJC  model  were  mandatory,  and  individuals  were  required  to  at- 
tend job  club  workshops  in  order  to  continue  receiving  their  TANF  grant.  (There  were  no 
additional  mandatory  TANF  participation  requirements  for  program  group  members  relative  to 
control  group  members.)  Because  participation  was  mandatory,  staff  did  not  have  to  recruit 
individuals,  as  they  would  have  needed  to  do  in  a voluntary  postemployment  program.  After 
attending  a program  orientation,  clients  completed  an  appraisal  interview  with  program  staff 
and  were  randomly  assigned  to  one  of  the  job  club  models.  The  job  clubs  did  not  use  financial 
incentives  for  participation  or  other  outcomes,  and  workshops  were  always  conducted  during 
standard  office  hours  at  the  program’s  offices.  Staff  did  not  attempt  to  engage  sample  members 
prior  to  the  scheduled  start  date  of  their  job  club  session,  and  if  they  did  not  show  up  for  their 
session,  they  were  rescheduled  for  another  session.  If  sample  members  continued  to  not  comply, 
they  were  sanctioned,  meaning  that  their  TANF  grant  could  be  reduced. 17 

Table  3.7  shows  that,  based  on  the  12-month  survey,  most  measures  of  sample  mem- 
bers’ contact  with  any  employment  program  staff  show  no  differences  between  the  groups.  One 
key  exception  is  that  the  EJC  survey  respondents  reported  a lower  rate  of  contact  than  the  TJC 
survey  respondents,  a 12  percentage  point  difference.  The  reason  for  this  is  not  clear.  The  two 
groups  did  not  differ  in  the  amount  of  time  that  they  received  TANF  in  the  year  following 
random  assignment,  and  thus  both  groups  had  equal  opportunities  for  having  contact  with  staff. 
The  difference  could  possibly  reflect  that  EJC  clients  found  jobs  more  quickly  than  TJC  clients 
(discussed  further  below),  resulting  in  a faster  exit  from  job  clubs  and  less  frequent  staff  contact. 


I7ln  California,  an  imposed  sanction  results  in  a reduction  in  the  amount  of  an  individual’s  TANF  grant.  In 
some  other  states,  an  imposed  sanction  can  result  in  eventual  forfeiture  of  the  full  TANF  grant  amount. 
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The  Employment  Retention  and  Advancement  Project 
Table  3.7 

Summary  of  Impacts: 


Los  Angeles  EJC 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

42.1 

53.9 

-118  *** 

0.004 

Average  number  of  contacts  with  staff 

7.3 

7.9 

-0.6 

0.701 

Talked  with  staff  in  past  4 weeks  (%) 

16.2 

20.1 

-3.9 

0.220 

Retention  services 

Participated  in  a job  search  activity  (%) 

70.8 

70.0 

0.9 

0.818 

Received  help  dealing  with  problems  on  the  job  (%) 

5.4 

5.1 

0.2 

0.899 

Advancement  services 

Received  help  with  career  assessment  (%) 

15.9 

12.0 

4.0 

0.169 

Received  help  finding  a better  job  while  working  (%) 

4.6 

6.6 

-2.0 

0.303 

Participated  in  an  education/training  activity3  (%) 

36.2 

41.5 

-5.3 

0.190 

Received  any  help  with  retention/advancement"  (%) 

21.9 

18.8 

3.1 

0.362 

Sample  size  (total  = 608) 

311 

297 

Cumulative  economic  outcomes  (Years  1-3) 

Employment  measures 

Ever  employed  (%) 

82.6 

82.8 

-0.2 

0.925 

Average  quarterly  employment  (%) 

49.5 

48.6 

0.9 

0.640 

Had  employment  spell  of  at  least  4 quarters  (%) 

56.3 

55.6 

0.7 

0.796 

Length  of  longest  employment  spell,  in  quarters 

5.2 

5.0 

0.2 

0.351 

Length  of  longest  unemployment  spell,  in  quarters 

5.3 

5.3 

0.0 

0.953 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

7,306 

7,325 

-18 

0.967 

Quarters  with  earnings  of  $3,500  or  more  (%) 

23.5 

23.9 

-0.4 

0.813 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  3 (%) 

Not  employed  in  one  or  both  years 

48.4 

48.0 

0.4 

0.882 

Earnings  decreased  by  more  than  $250 

15.3 

11.7 

3.7  * 

0.065 

Earnings  changed  by  less  than  $250 

4.1 

3.8 

0.3 

0.764 

Earnings  increased  by  $250  or  more 

32.1 

35.8 

-3.7 

0.168 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

4,037 

3,926 

111 

0.443 

Average  annual  food  stamps  received  ($) 

2,213 

2,125 

88 

0.240 

Average  annual  income3  ($) 

13,556 

13,376 

181 

0.664 

Sample  size  (total  = 1,183) 

598 

585 

(continued) 
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Table  3.7  (continued) 


SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

This  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

This  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


Initial  Job  Preparation  and  Placement  Activities 

The  Los  Angeles  EJC  model’s  step-down  approach  used  job  search  activities  targeted  at 
a specific  range  of  jobs  within  an  occupational  field  of  interest  defined  by  a client.  The  overall 
objective  of  the  EJC  model  was  to  connect  job-seekers  with  positions  in  a career  field  of  their 
choosing  rather  than  instructing  them  to  take  any  available  job. 

For  the  first  week  of  the  five-week  workshop,  the  EJC  model  involved  clients  in  devel- 
oping a career  plan  that  identified  their  occupational  field  of  interest  and  various  levels  of  jobs 
within  that  field.  Clients  identified  a “quality-of-life  goal”  that  was  the  individual’s  ultimate 
career  objective,  which  usually  required  additional  experience  or  credentials.  Clients  also 
identified  a list  of  “targeted  jobs”  (the  highest-paying  jobs  in  one’s  career  field,  given  current 
education  or  work  history)  and  a list  of  “promotable”  jobs  that  could  lead  to  a targeted  job.  In 
addition,  the  EJC  workshop  included  activities  to  impart  basic  job  search  skills,  such  as  resume 
preparation  and  interviewing  practice. 

During  the  three  subsequent  weeks,  EJC  clients  refined  their  career  development  plans 
and  conducted  the  step-down  targeted  job  search  activities.  First,  individuals  attempted  to 
become  employed  in  the  highest-paying  job  in  their  field  of  interest  that  they  could  access  (their 
targeted  job).  If  they  did  not  find  a targeted  job  during  the  second  week  of  job  search,  they 
expanded  the  search  to  include  both  targeted  and  promotable  positions  during  the  third  week. 
Lastly,  if  clients  were  still  unemployed  by  the  fourth  week  of  job  search,  then  they  aimed  to  find 
a skill-building  job  (any  part-time  position)  that  could  improve  their  skills  if  combined  with 
enrollment  in  education  or  training.  The  fifth  week  was  voluntary  and  consisted  of  individua- 
lized job  search  assistance  for  those  who  wanted  additional  help  finding  work.  Overall,  field 
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research  suggests  that  staff  successfully  implemented  different  activities  in  the  EJC  and  the  TJC 
workshops  and  conveyed  distinctly  different  messages  about  the  types  of  jobs  to  seek. 

The  nature  of  the  two  groups’  job  club  experiences  was  expected  to  differ,  and  the  EJC 
group,  compared  with  the  TJC  group,  was  expected  to  have  longer  potential  stays  in  the  job 
clubs  as  well  (five  weeks  compared  with  three  weeks).  However,  according  to  the  12-month 
survey,  the  EJC  and  TJC  sample  members  attended  job  clubs  at  similar  rates  and  for  similar 
amounts  of  time.  About  60  percent  of  both  groups  participated  in  group  job  search  activities  in 
the  first  year  (Appendix  Table  A.  12),  and  each  group  attended  for  about  2.5  weeks  in  the  first 
year  (not  shown).18 

Despite  the  efforts  of  EJC  staff,  EJC  clients  did  not  retain  the  message  of  developing  a 
long-term  career  development  plan  and  finding  jobs  in  their  occupational  field  of  interest. 
Findings  from  the  12-month  survey  show  that  the  overall  message  recalled  by  the  majority  of 
EJC  clients  a year  after  they  entered  the  model  was  that  they  should  quickly  find  a job,  and  there 
were  few  differences  between  what  the  EJC  and  the  TJC  sample  members  recalled  regarding 
the  types  of  messages  they  received  (not  shown). 19  This  could  reflect  that  exposing  clients  to  a 
message  that  is  more  nuanced  than  the  immediate  “work-first”  message  is  difficult  within  an 
agency  that  has  a strong  employment  focus.  Also,  according  to  field  research,  there  were 
differences  among  EJC  facilitators  regarding  how  closely  they  followed  the  progression  of  job 
types  — quality-of-life,  targeted,  promotable,  and  skill-building  — and  this  also  may  have 
weakened  the  EJC  message. 

Retention  Services 

The  Los  Angeles  EJC  model  stressed  the  importance  of  helping  people  find  work  in 
their  field  of  interest  as  a key  strategy  for  increasing  employment  retention  rates,  but  it  did  not 
provide  any  specific  services  in  this  area.  Both  the  EJC  and  the  TJC  model  addressed  retention 
issues  in  indirect  ways,  such  as  encouraging  clients  to  leam  and  meet  employers’  expectations 
regarding  work  site  behavior  (especially  interpersonal  relationships  between  supervisors  and 
employees)  and  ensuring  that  clients  made  sufficiently  reliable  arrangements  for  their  child  care 
and  transportation  needs  in  preparation  for  employment  (Appendix  Table  A.  12).  And  staff  in 
both  job  clubs  made  no  efforts  to  stay  in  contact  with  employed  participants.  Not  surprisingly, 
Table  3.7  shows  no  differences  between  the  two  groups  in  terms  of  participation  in  employment 
retention  services. 


18Navarro,  Azurdia,  and  Hamilton  (2008). 
19Navarro,  Azurdia,  and  Hamilton  (2008). 
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Advancement  Services 


The  career  development  activities  — especially  the  formulation  of  the  career  plan  and 
the  lists  of  targeted  and  promotable  jobs  that  clients  attempted  to  find  — were  critical  elements 
of  the  Los  Angeles  EJC  model.  The  career  plan,  in  particular,  acted  as  a repository  of  all  the 
steps  that  individuals  needed  to  undertake  in  order  to  obtain  higher-quality  jobs.  In  contrast,  TJC 
did  not  offer  any  career  development  services  and  was  not  designed  to  discuss  advancement  in  a 
specific  manner.  Yet  there  is  no  difference  between  the  EJC  and  TJC  sample  members  in  client- 
reported  rates  of  participation  in  career  assessment  services  (Table  3.7). 

If  unable  to  find  a desirable  full-time  job  in  the  five-week  program,  the  EJC  clients  had 
quicker  access  to  education  and  training  than  their  counterparts  in  the  TJC  model.  EJC  clients 
could  undergo  a vocational  assessment  near  or  at  the  end  of  the  workshop,  which  was  required 
in  order  to  be  referred  to  education  or  training.  EJC  staff  also  encouraged  clients  to  blend  part- 
time  work  with  their  participation  in  education  or  training.  However,  the  survey  data  show  that 
TJC  and  EJC  group  members  enrolled  in  education  and  training  at  the  same  rates  (Table  3.7). 

Employer  Linkages 

The  EJC  model  included  only  minimal  efforts  to  involve  employers  in  job  placement  or 
career  development  activities.  To  support  EJC  clients’  job  search  efforts  throughout  the  five- 
week  workshop,  job  developers  matched  clients  to  specific  job  openings,  coordinated  weekly 
employer  recruitments,  and  referred  clients  to  job  fairs. 

Los  Angeles  EJC  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Los  Angeles  EJC  program  can  be  summarized  as  follows: 

• Los  Angeles  EJC  did  not  produce  any  changes  in  the  main  measures  of  em- 
ployment, employment  retention,  earnings,  or  advancement,  compared  with 
the  TJC  group,  in  the  cumulative  three-year  follow-up  period. 

• EJC  did  not  have  any  effect  on  the  amount  of  TANF  or  food  stamp  assis- 
tance received  or  on  the  average  annual  income  during  this  period. 

Control  Group  Outcomes 

Average  earnings  for  TJC  (control)  group  members  in  the  Los  Angeles  EJC  ERA  study 
were  the  highest  of  all  the  control  groups  examined  in  this  chapter,  due  to  a substantial  increase 
in  earnings  over  the  follow-up  period.  The  average  quarterly  employment  rate  was  under  50 
percent,  and  average  earnings  were  about  $7,300  per  year  for  TJC  group  members  over  the 
three-year  follow-up  period  (Table  3.7).  Average  earnings  almost  doubled  over  the  follow-up 


period  — without  a corresponding  increase  in  employment20  — which  suggests  that  there  was 
some  advancement  over  time  in  the  TJC  group  (Appendix  Table  A.  13).  While  most  of  the  TJC 
group  received  at  least  one  TANF  payment  in  the  first  follow-up  year,  57  percent  of  that  group 
received  at  least  one  TANF  payment  in  the  third  year  of  follow-up  (Appendix  Table  A.  13). 

Employment  and  Employment  Retention 

In  the  cumulative  three-year  follow-up  period,  Los  Angeles  EJC  did  not  lead  to  gains, 
relative  to  the  TJC  group,  in  any  of  the  main  measures  of  employment  or  employment  retention 
(Table  3.7).  However,  EJC  did  produce  an  increase  in  the  percentage  of  program  group  mem- 
bers who  were  continuously  employed,  an  increase  of  about  5 percentage  points  (Appendix 
Table  A.  14).  EJC  also  increased  the  length  of  time  that  individuals  were  employed  at  the  jobs 
held  in  the  first  quarter  after  random  assigmnent.  These  findings  could  indicate  a positive  effect 
of  EJC  on  a small  group  of  people  — perhaps  for  people  who  were  able  to  find  a job  in  their 
career  of  interest. 

Earnings  and  Advancement 

EJC  did  not  produce  any  increases  in  any  of  the  main  measures  of  earnings  or  ad- 
vancement, compared  with  what  was  seen  in  the  TJC  group  (Table  3.7). 

Public  Assistance  and  Average  Annual  Income 

In  the  cumulative  three-year  follow-up  period,  EJC  had  no  effect  on  the  amount  of 
TANF  benefits  received,  on  the  amount  of  food  stamp  assistance  received,  or  on  the  average 
annual  income  (Table  3.7). 

Los  Angeles  EJC  Program:  Summary  and  Conclusions 

Past  research  showed  that  the  Traditional  Job  Club  (TJC)  model  emphasizing  imme- 
diate entry  into  any  job  was  effective  in  increasing  employment  and  earnings  for  welfare 
applicants  and  recipients,  relative  to  not  requiring  participation  in  job  clubs.  Yet  the  model  was 
not  found  to  be  effective  in  helping  people  to  retain  or  advance  in  jobs.  In  an  attempt  to  improve 
job  quality  and  increase  employment  retention  and  advancement  for  unemployed  TANF 
recipients,  the  Los  Angeles  EJC  ERA  model  made  alterations  to  this  traditional  job  club  model. 
Specifically,  the  EJC  model  added  career  development  activities,  promoted  the  use  of  targeted 
job  searches  that  encouraged  individuals  to  seek  jobs  in  their  career  of  interest,  and  extended  the 


20Because  people  who  are  not  employed  in  a given  quarter  have  earnings  of  zero,  and  because  these  zeros 
are  included  in  average  earnings,  average  earnings  could  increase  over  time  as  employment  increased  even  if 
earnings  did  not  increase  at  all  for  people  who  were  employed. 
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length  of  the  activity  from  three  to  five  weeks,  under  the  theory  that  this  would  help  individuals 
find  jobs  that  they  would  retain  and  use  as  a basis  for  advancement. 

Staff  implemented  different  types  of  activities  in  the  EJC  and  the  TJC  workshops  and 
conveyed  distinctly  different  messages  about  the  types  of  jobs  that  individuals  were  to  seek. 
EJC  staff  worked  closely  with  clients  to  identify  their  field  of  interest  and  the  potential  barriers 
they  faced  in  acquiring  a job  in  their  ideal  field,  and  then  they  tailored  job  searches  to  clients’ 
interests.  In  contrast,  TJC  staff  urged  clients  to  seek  and  accept  any  type  of  job. 

Despite  the  different  messages  conveyed  by  staff  to  individuals  in  the  EJC  and  TJC 
groups,  the  overall  message  that  was  recalled  one  year  later  by  both  EJC  and  TJC  group 
members  was  that  they  should  find  a job  quickly.  In  addition,  EJC  and  TJC  group  members 
attended  job  clubs  for  similar  lengths  of  time,  even  though  the  EJC  workshops  were  designed  to 
last  two  weeks  longer.  No  differences  were  expected  between  the  two  groups  in  the  receipt  of 
retention  and  advancement  services  as  measured  in  the  12-month  survey,  and  none  were  found. 

While  EJC  did  increase  the  average  length  of  time  that  people  were  employed  at  the 
jobs  they  held  in  the  first  quarter  after  random  assignment,  compared  with  the  average  time  for 
individuals  served  by  TJC,  it  did  not  generate  any  changes  in  the  main  measures  of  employ- 
ment, employment  retention,  earnings,  or  advancement  in  the  cumulative  three-year  follow-up 
period.  It  appears  that,  despite  the  efforts  of  EJC  staff,  the  overall  message  that  was  recalled  by 
the  majority  of  EJC  survey  respondents  a year  after  they  entered  the  study  was  that  they  should 
quickly  find  a job.  This  suggests  that  the  “work- first”  mandate  in  the  overall  program  in  which 
the  EJC  model  was  embedded  may  have  been  more  influential  than  the  job  club  message, 
content,  or  curriculum.  Exposing  clients  to  a message  that  is  more  nuanced  than  “find  a job 
quickly”  may  be  difficult  within  a work-focused  welfare  agency.  ERA  program  group  members 
may  have  also  found  it  difficult  to  leam  about  and  obtain  “targeted”  or  “promo table”  jobs  in 
their  fields  of  interest,  particularly  in  the  time  frame  available.  And,  even  for  those  who  did  find 
such  jobs,  the  jobs  may  not  have  paid  well  or  may  have  otherwise  not  been  “good”  jobs  (with 
full-time  hours,  benefits,  and  so  on).  Overall,  although  field  research  shows  that  the  EJC  model 
was  implemented  largely  as  it  was  designed,  it  did  not  prove  to  be  more  effective  than  the  TJC 
model  at  promoting  employment  retention  and  advancement. 
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Salem  ERA  Findings 


Salem  ERA  at  a Glance 

• Served  unemployed  TANF  applicants  who  were  required  to  search  for  employment 

• Sought  to  provide  individualized  retention  and  advancement  services  both  before  and 
after  employment 

• In  practice,  provided  preemployment  services  but  had  difficulty  implementing  postem- 
ployment services 

• Was  compared  with  a control  group  that  participated  in  Oregon’s  standard  welfare -to- 
work  program 

• Resulted  in  small-to-moderate  increases  in  participation  in  employment  retention  and 
advancement  services 

• Produced  no  improvements  in  employment  retention  or  advancement 


Salem  ERA  Test:  Introduction 

Operating  from  May  2002  through  June  2005,  the  Salem  ERA  program  targeted  TANF 
applicants  who  were  unemployed.  The  program  began  with  job  search  and  placement  services 
that  focused  on  employment  retention  and  advancement;  once  individuals  were  employed,  the 
Salem  ERA  program  intended  to  provide  further  services  to  promote  employment  retention  and 
advancement. 

Origins  of  the  Test 

The  Oregon  and  Salem  Departments  of  Human  Services  (DHS)  were  actively  engaged 
in  workforce  retention  and  advancement  issues  prior  to  designing  the  ERA  program  in  2002 
and,  as  part  of  this  effort,  had  concluded  that  a strong  client-staff  relationship  was  integral  to 
clients’  participation  in  services  and  employment  retention  and  advancement.  Beginning 
services  prior  to  employment  was  considered  important  in  building  this  relationship.  The 
importance  of  engaging  employers  was  also  identified. 

The  Salem  ERA  program  was  designed  to  be  different  from  the  standard  welfare-to- 
work  program  that  was  offered  to  control  group  members.  It  differed  in  its  emphasis  on  helping 
clients  identify  and  work  toward  a job  in  their  field  of  interest,  its  focus  on  customer  service,  the 
location  of  the  program  in  a local  One-Stop  Center  rather  than  the  welfare  office,  and  the 
availability  of  assistance  and  services  after  employment  was  secured. 

Those  who  were  assigned  to  the  control  group  participated  in  the  standard  welfare-to- 
work  program  providing  preemployment  job  search  services.  The  control  group  services 
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embodied  a “work-first”  philosophy  and  essentially  ended  once  clients  found  jobs  and  were 
determined  no  longer  to  be  eligible  for  TANF  assistance. 


Labor  Market  Context 

The  Salem  ERA  program  operated  in  a difficult  environment.  At  the  time  of  random  as- 
signment, the  area’s  labor  market  was  still  suffering  the  effects  of  the  2001  recession,  with 
rising  unemployment  rates  and  stagnant  job  growth.  During  the  study  period,  the  unemploy- 
ment rate  in  the  Salem  area  continued  to  increase  through  2003.  Further,  the  State  of  Oregon, 
Salem’s  largest  employer,  instituted  a hiring  freeze  due  to  budget  shortfalls,  and  this  affected  the 
types  of  job  opportunities  available  to  program  group  members  as  well  as  the  program’s  staffing 
capacity.  At  the  same  time,  funds  for  the  TANF  program  were  being  reallocated  to  support 
other  programs.  As  the  area’s  labor  market  began  to  recover  in  2004,  unemployment  declined, 
and  job  growth  increased  modestly,  but  unemployment  still  continued  to  be  above  the  national 
rate.  Thus,  for  most  of  the  Salem  ERA  program’s  operating  period,  labor  market  conditions 
created  a challenging  environment  for  sample  members  looking  for  work,  though  there  were 
some  small  improvements  toward  the  end  of  the  follow-up  period. 

Target  Population 

The  Salem  ERA  program  targeted  low-income  TANF  applicants.  Table  3.2  shows  se- 
lected characteristics  of  the  sample  members  at  the  time  of  random  assignment.  Sixty-nine 
percent  of  the  Salem  ERA  sample  had  at  least  a high  school  diploma  or  a General  Educational 
Development  (GED)  certificate,  and  45  percent  had  worked  more  than  24  months  out  of  the  last 
three  years  (both  are  the  highest  rates  for  sites  within  this  target  group).  Nearly  two-thirds  of 
employed  Salem  ERA  sample  members  earned  $7.00  per  hour  or  more  at  the  jobs  they  held  at 
the  time  of  random  assignment.  This  proportion  is  high  compared  with  other  sites  in  this  target 
group,  likely  because  Oregon’s  minimum  wage  ($6.50  to  $6.90  per  hour  during  most  of  the 
time  the  program  was  operating)  was  higher  than  the  national  minimum  hourly  wage  at  the  time 
($5.15). 


Significance  of  the  Test 

The  Salem  ERA  test  examines  whether  mandatory  preemployment  and  voluntary  post- 
employment services  can  increase  job  placement,  job  quality,  employment  retention,  and 
advancement  in  the  labor  market  for  TANF  applicants,  compared  with  a program  limited  to 
mandatory  preemployment  services.  Chart  3.3  highlights  important  differences  between  the 
programs  for  the  ERA  group  and  the  control  group  in  Salem. 
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The  Employment  Retention  and  Advancement  Project 
Chart  3.3 

Program-Control  Group  Differences: 

Salem  ERA  Test 


ERA  Group 

Control  Group 

Goals 

• To  promote  job  placement,  employment 
retention,  and  advancement  among  unemployed 
TANF  applicants 

• To  promote  job  placement  among 
unemployed  TANF  applicants 

Staffing 

• Initially  team-based  and  later  specialized  staff 
structure 

• Staff  routinely  able  to  meet  clients  off-site 

• ERA  staff  located  at  One-Stop  Center 

• Generalist  and  not  team-based  staff 
structure 

• Staff  unable  to  meet  clients  off-site 

• Staff  located  at  DHS  office 

Staff-client 

engagement 

• During  preemployment,  staff-to-client  contact 
2-3  times  per  week 

• Development  of  employment  plan 

• During  preemployment,  staff-to-client 
contact  once  per  week 

• No  employment  plan  developed 

Job 

preparation 
and  placement 

• Job  search  assistance,  including  pre- 
employment workshops  focused  on  retention 
and  career  advancement 

• Staff  assistance  with  and  easy  access  to  One- 
Stop  Center  job  search  resources 

• Job  search  assistance  focused  on  quick 
placement  with  no  regard  for  retention  and 
career  advancement 

• No  staff  assistance  with  One-Stop  Center 
job  search  resources 

Advancement 

services 

• Preemployment  workshops  with  career  interest 
assessments  and  career  path  activities 

• No  advancement  services 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


Salem  ERA  Program:  Features 

Program  Resources  and  Organization 

Building  on  an  already  strong  working  relationship  among  the  welfare,  workforce,  and 
community  college  systems,  the  Salem  ERA  program  was  a collaborative  effort  between  the 
Oregon  Department  of  Human  Services  (DHS)  and  Chemeketa  Community  College  (Chemeke- 
ta)  — the  same  organizations  that  operated  the  program  for  the  control  group.  Staff  from  both 
organizations  delivered  ERA  services  at  the  Winema  Career  Center  (the  One-Stop  Center  in 
Salem),  located  on  the  Chemeketa  campus. 
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Staffing 

Each  ERA  client  worked  with  a DHS  staff  member  and  with  a training  and  employment 
specialist  employed  by  Chemeketa.  The  DHS  staff  were  responsible  for  all  eligibility  issues  for 
TANF,  food  stamps,  child  care,  transportation,  and  the  Oregon  Health  Plan.  The  training  and 
employment  specialists  were  in  charge  of  the  client’s  job  search,  employment  retention,  and 
career  advancement.  Unlike  the  control  group  services  that  were  housed  at  DHS,  the  Winema 
Career  Center  was  designated  as  the  physical  home  for  the  Salem  ERA  program  and  for  both 
DHS  staff  and  Chemeketa  training  and  employment  specialists.  Although  the  two  types  of  ERA 
staff  had  some  distinct  responsibilities,  they  were  presented  to  clients  as  a team,  any  one  of 
whom  could  be  approached  for  help.  Colocation  of  staff  and  the  lack  of  specialization  along 
pre-  and  postemployment  lines  were  intended  to  make  services  seamless  and  the  ERA  program 
more  customer  friendly. 

The  Salem  ERA  program  experienced  high  staff  turnover  early  in  the  study  period  and 
was  unable  to  fill  its  staff  vacancies  due  to  the  statewide  hiring  freeze.  As  a result,  while  it  was 
intended  that  staff  would  work  with  employed  and  unemployed  clients,  as  caseloads  grew  and 
staffing  levels  dropped,  staff  focused  primarily  on  preemployment  services,  such  as  eligibility 
issues  and  individual  crises.  Near  the  end  of  the  study  period,  management  responded  to 
workload  challenges  by  specializing  some  of  the  ERA  staff  responsibilities.  Program  managers 
designated  a training  and  employment  specialist  and  one  other  staff  member  as  the  career 
advancement  team,  which  focused  on  the  development  of  postemployment  services. 

ERA  staff  were  expected  to  accommodate  clients’  schedules  and  locations,  meeting 
clients  at  sites  other  than  their  offices  at  the  Career  Center.  The  training  and  employment 
specialists  routinely  made  an  effort  to  meet  clients  during  their  lunch  hour  (often  buying  them 
lunch)  and  at  their  job  sites.  The  ERA  staff  had  high  levels  of  contact  with  clients  during  the 
preemployment  job  search  period,  partly  because  most  clients  were  required  to  meet  with  staff 
once  a week.  Moreover,  because  the  ERA  program  offered  workshops  four  days  per  week, 
participants  often  dropped  by  informally  after  attending  a workshop,  and  they  accessed  job 
search  services  at  the  Career  Center. 

The  original  ERA  staff  completed  a 120-hour  career  development  facilitation  training 
course  at  Chemeketa  Community  College  in  addition  to  other  training  that  was  focused  on 
marketing  and  client  engagement  strategies.  Unfortunately,  due  to  the  high  level  of  staff 
turnover,  this  knowledge  was  not  retained. 

Cultural  differences  between  staff  from  the  two  organizations  providing  ERA  services 
created  some  operational  difficulties.  Welfare  and  community  college  staff  entered  the  Salem 
ERA  program  with  different  philosophies  about  program  policies  and  goals,  and  they  had 
different  expectations  for  the  program.  Even  though  ERA  program  managers  worked  well 
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together,  they  were  stretched  across  many  commitments,  which  limited  their  ability  to  address 
these  philosophical  differences. 

Staff-Client  Engagement 

TANF  participation  requirements  were  the  same  for  individuals  in  the  program  group 
and  those  in  the  control  group:  participation  in  the  preemployment  phase  was  mandatory,  but 
participation  after  employment  was  voluntary.  (ERA-specific  activities  did  not  introduce  any 
additional  mandatory  TANF  participation  requirements  for  program  group  members  relative  to 
control  group  members.)  As  mandated  by  the  state,  all  welfare  applicants,  unless  exempt,  were 
required  to  participate  in  job  search  activities  that  occurred  concurrently  with  applicants’ 
eligibility  determination.  When  they  first  applied  for  welfare,  individuals  were  assigned  either  to 
the  ERA  program  group  or  to  the  control  group,  which  provided  the  standard  welfare-to-work 
job  search  and  preparation  services. 

Those  assigned  to  the  ERA  program  were  referred  to  an  intake  meeting  and  orientation 
at  the  Winema  Career  Center.  During  these  meetings,  ERA  program  staff  invested  considerable 
time  and  effort  into  marketing  the  ERA  program  differences  from  regular  welfare-to-work 
services,  stressing  the  team  approach,  enhanced  customer  service,  and  ongoing  services  after 
job  placement.  After  the  orientation,  clients  met  one-on-one  with  ERA  staff  who  further 
explained  the  goals  and  benefits  of  the  program.  Subsequently,  ERA  staff  worked  closely  with 
clients  to  develop  an  individual  Personal  Development  Plan,  which  outlined  the  client’s  short- 
and  long-term  career  goals.  These  goals  were  revisited  and  updated  over  the  course  of  the 
program  and  even  after  the  client  became  employed.  Staff  spent  significant  time  assisting  clients 
with  eligibility  issues  and  resolving  employment  barriers,  such  as  concerns  about  child  care, 
housing,  or  transportation. 

ERA  clients  were  required  to  meet  with  staff  once  a week,  and  they  also  would  see  staff 
when  they  attended  preemployment  workshops.  Although  the  ERA  program  did  not  affect  the 
proportion  of  clients  who  had  any  contact  with  staff  from  an  employment  program,  the  program 
did  increase  the  average  number  of  contacts  made  in  the  first  12  months.  Table  3.8  shows  that 
program  group  members  had  an  average  of  22  contacts  in  the  year  after  random  assignment,  an 
increase  of  nearly  7 contacts  over  the  control  group  average  in  that  time  frame. 

Initial  Job  Preparation  and  Placement  Sen/ices 

Like  all  the  programs  that  served  unemployed  TANF  recipients,  job  preparation  and 
placement  services  were  a primary  component  of  the  Salem  ERA  program,  which  created  a set 
of  preemployment  workshops  that  focused  on  future  employment  retention  and  career  paths. 
These  workshops  were  specifically  designed  to  address  longer-term  career  interests  and  to  focus 
clients  on  their  “dream  job.”  Program  staff  reported  some  difficulties  motivating  clients  to 
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The  Employment  Retention  and  Advancement  Project 
Table  3.8 

Summary  of  Impacts: 

Salem 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

84.9 

79.0 

5.9 

0.197 

Average  number  of  contacts  with  staff 

22.0 

15.4 

6.6  ** 

0.042 

Talked  with  staff  in  past  4 weeks  (%) 

41.7 

41.0 

0.7 

0.907 

Retention  services 

Participated  in  a job  search  activity  (%) 

77.6 

70.3 

7.2 

0.166 

Received  help  dealing  with  problems  on  the  job  (%) 

15.2 

4.7 

10  5 *** 

0.003 

Advancement  services 

Received  help  with  career  assessment  (%) 

16.7 

9.9 

6.8  * 

0.098 

Received  help  finding  a better  job  while  working  (%) 

9.2 

3.4 

5.9  ** 

0.036 

Participated  in  an  education/training  activity3  (%) 

28.6 

32.8 

-4.1 

0.462 

Received  any  help  with  retention/advancement0  (%) 

34.1 

20.4 

13.8  ** 

0.011 

Sample  size  (total  = 300) 

152 

148 

Cumulative  economic  outcomes  (Years  1-3) 

Employment  measures 

Ever  employed  (%) 

81.3 

80.4 

0.8 

0.666 

Average  quarterly  employment  (%) 

42.4 

44.9 

-2.4 

0.153 

Had  employment  spell  of  at  least  4 quarters  (%) 

47.5 

49.2 

-1.7 

0.494 

Length  of  longest  employment  spell,  in  quarters 

4.3 

4.6 

-0.3 

0.110 

Length  of  longest  unemployment  spell,  in  quarters 

5.9 

5.8 

0.2 

0.370 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

5,258 

5,519 

-260 

0.425 

Quarters  with  earnings  of  $3,500  or  more  (%) 

16.7 

18.2 

-1.6 

0.234 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  3 (%) 

Not  employed  in  one  or  both  years 

56.3 

52.9 

3.4 

0.163 

Earnings  decreased  by  more  than  $250 

11.0 

11.4 

-0.4 

0.786 

Earnings  changed  by  less  than  $250 

3.0 

4.0 

-1.0 

0.283 

Earnings  increased  by  $250  or  more 

29.6 

31.6 

-2.0 

0.395 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

1,602 

1,304 

293  *** 

0.001 

Average  annual  food  stamps  received  ($) 

2,305 

2,212 

93 

0.168 

Average  annual  income3  ($) 

9,166 

9,035 

131 

0.690 

Sample  size  (total  = 1,504) 

742 

762 

(continued) 
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Table  3.8  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

“This  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


attend  all  the  workshops  available,  but  they  encouraged  clients  to  identify  and  work  toward  a 
job  of  interest  or  in  their  “field  of  fascination.”  In  addition  to  the  workshops,  individuals  in  the 
ERA  program  received  one-on-one  assistance  from  program  staff  to  address  employment 
barriers  and  provide  support  services. 

While  the  services  at  the  Career  Center  were  also  available  to  individuals  in  the  control 
group,  the  ERA  clients  had  a stronger  and  more  immediate  access  to  these  job-related  resources 
because  their  program  was  located  at  the  Career  Center.  The  Salem  ERA  staff  assisted  clients  in 
accessing  these  services,  while  individuals  in  the  control  group  were  less  likely  to  know  about 
or  use  them. 

Overall,  the  Salem  ERA  program  engaged  a large  majority  of  its  clients  in  program  ser- 
vices, although  control  group  members  had  similarly  high  levels  of  participation.  As  shown  in 
Table  3.8,  about  three-quarters  of  both  groups  participated  in  some  type  of  job  search  in  the  first 
year  of  follow-up.  However,  participation  in  a particular  type  of  job  search  — group  job  search 
or  job  club  — was  higher  for  the  program  group  than  for  the  control  group,  and  program  group 
members  stayed  in  these  activities  longer.  Sixty-three  percent  of  program  group  members 
participated  in  a group  job  search  or  job  club,  a rate  that  is  13  percentage  points  greater  than  the 
control  group  rate  (Appendix  Table  A.  16).  On  average,  program  group  members  participated  in 
these  activities  for  15  weeks  in  the  first  year,  compared  with  9 weeks  among  control  group 
members  (Appendix  Table  A.  16). 


97 


Retention  Services 


The  strength  of  the  Salem  ERA  program’s  preemployment  services  — particularly  the 
emphasis  on  career  advancement  and  the  staffs  enhanced  customer  service  — was  intended  to 
help  clients  who  found  employment  stay  engaged  in  the  program  and  participate  in  its  postem- 
ployment services.  But  the  postemployment  services  were  also  to  include  immediate  and 
ongoing  follow-up  with  newly  working  clients,  in  order  to  revisit  goals,  address  any  barriers  to 
employment  retention,  and  make  referrals  to  appropriate  education  or  training  activities  or  to 
jobs  that  paid  more.  In  practice,  however,  staff  were  unable  to  provide  these  postemployment 
services  consistently,  although  their  ability  to  do  so  did  increase  toward  the  end  of  the  study 
period.  Overall,  only  15  percent  of  the  program  group  reported  having  received  help  dealing 
with  problems  on  the  job,  but  this  was  1 1 percentage  points  greater  than  the  control  group  rate 
(Table  3.8). 

Advancement  Sen/ices 

For  most  of  the  study  period,  the  primary  advancement  strategy  in  the  Salem  ERA  pro- 
gram centered  on  the  preemployment  workshops  that  were  infused  with  messages  and  activities 
related  to  career  advancement.  While  the  program  had  intended  to  involve  education  and 
training  providers  and  to  work  with  clients  to  explore  education  and  training  options,  in  practice, 
these  activities  occurred  on  a very  limited  basis. 

Only  when  the  program  was  nearing  its  end  and  random  assignment  had  stopped  did 
staff  have  the  time  and  resources  to  turn  attention  to  working  clients  and  the  program’s  postem- 
ployment objectives.  At  this  point,  staff  shifted  from  the  team-based  approach  into  specialized 
positions  and  focused  on  advancement  through  regular  meetings  with  working  clients  and 
“Career  Night,”  a forum  in  which  working  clients  convened  to  discuss  issues  that  could  affect 
their  employment  retention  and  prospects  for  advancement.  Despite  direct  mailing  and  reminder 
phone  calls,  however,  attendance  at  these  workshops  was  low. 

Overall,  34  percent  of  the  program  group  reported  having  received  help  with  retention 
or  advancement  services  — 14  percentage  points  greater  than  the  control  group  rate  (Appendix 
Table  A.  16).  Twenty-three  percent  of  program  group  members,  or  nearly  double  the  rate  of 
control  group  members,  reported  that,  while  working,  they  participated  in  an  employment 
activity  (primarily,  group  or  individual  job  search)  (Table  3.8).  Although  the  ERA  program  did 
increase  the  receipt  of  these  services,  the  overall  level  of  participation  was  still  lower  than 
expected. 

The  lower-than-expected  level  of  participation  in  postemployment  services  may  have 
reflected  several  factors.  First,  clients’  availability  and  follow-through  opportunities  were 
limited,  given  the  voluntary  nature  of  the  postemployment  services.  Because  of  the  relatively 


98 


high  minimum  wage  in  Oregon,  those  who  found  employment  were  generally  ineligible  for 
cash  assistance  and  were  then  no  longer  required  to  participate  in  program  services,  which 
weakened  their  link  to  the  program.  Second,  many  staff  reported  their  own  lack  of  skills  and 
experience  in  delivering  advancement  services.  They  struggled  to  understand  how  to  develop 
advancement  services  in  general,  the  strategies  and  methods  involved,  and  how  to  approach 
employers  and  clients  regarding  advancement.  Finally,  higher  caseloads  than  anticipated  made 
it  difficult  for  staff  to  focus  on  postemployment  services. 

Employer  Linkages 

While  a key  goal  of  the  Salem  ERA  program  was  to  engage  employers,  this  objective 
never  materialized,  and  the  program  had  little  direct  contact  with  employers.  In  part,  the  state’s 
hiring  freeze  affected  such  linkages,  since  the  public  sector  was  expected  to  be  a major  employ- 
er of  ERA  participants.  In  addition,  staff  reported  that  they  were  unsure  of  how  to  approach 
employers  and  that  most  clients  did  not  want  program  staff  involved  in  their  work  relationships. 

Salem  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Salem  ERA  program  can  be  summarized  as  follows: 

• The  Salem  ERA  program  did  not  produce  any  improvements,  relative  to  the 
control  group,  in  the  main  measures  of  employment,  employment  retention, 
or  earnings  in  the  cumulative  three-year  follow-up  period.  There  is  also  no 
evidence  that  ERA  led  to  advancement  gains,  compared  with  the  level  of  ad- 
vancement seen  in  the  control  group. 

• ERA  increased  the  amount  of  TANF  received  but  did  not  have  any  effect  on 
the  amount  of  food  stamp  assistance  received  or  on  average  annual  income, 
compared  with  the  control  group  rates. 

Control  Group  Outcomes 

Average  quarterly  employment  for  control  group  members  in  Salem  was  low  — about 
45  percent  — and  average  earnings  for  the  control  group  were  $5,500  per  year  over  the  three- 
year  follow-up  period  (Table  3.8).  Both  of  these  outcomes  improved  over  the  follow-up  period 
(Appendix  Table  A.  17).  While  over  half  of  the  control  group  received  at  least  one  TANF 
payment  in  the  first  follow-up  year,  in  the  third  year  of  follow-up,  only  27  percent  of  the  control 
group  received  at  least  one  TANF  payment  (Appendix  Table  A.  17). 
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Employment  and  Employment  Retention 

The  Salem  ERA  program  did  not  generate  any  positive  changes,  relative  to  the  control 
group,  in  the  main  measures  of  employment  or  employment  retention  in  the  cumulative  three- 
year  follow-up  period  (Table  3.8).  However,  some  additional  measures  (shown  in  Appendix 
Table  A.  18)  suggest  that  ERA  may  have  had  a negative  effect  on  employment  retention,  relative 
to  the  control  group.  ERA  produced  decreases  in  the  average  number  of  quarters  in  the  first 
employment  spell  and  in  the  percentage  of  people  continuously  employed  during  the  three- year 
follow-up  period.  ERA  also  decreased  average  quarterly  employment  by  almost  4 percentage 
points  in  the  third  year  of  follow-up  (Appendix  Table  A.17).  As  discussed  below,  ERA  also  led 
to  increases  in  the  receipt  of  public  assistance. 

Earnings  and  Advancement 

There  is  no  evidence  that  the  Salem  ERA  program  produced  any  changes,  either  posi- 
tive or  negative,  in  any  of  the  measures  of  earnings  or  advancement  for  the  program  group, 
compared  with  the  control  group. 

Public  Assistance  and  Average  Annual  Income 

Over  the  three-year  follow-up  period,  the  ERA  program  in  Salem  increased  annual 
TANF  payments  by  an  average  of  about  $300  over  the  levels  for  the  control  group  (Table  3.8). 
The  program  did  not  increase  food  stamp  receipt  or  average  annual  income  for  program  group 
members,  relative  to  control  group  members,  in  the  cumulative  follow-up  period.  The  impacts 
on  TANF  receipt  were  largely  concentrated  in  the  first  two  years  of  follow-up,  when  the 
program  increased  the  amount  of  TANF  assistance  received  by  20  percent  and  32  percent  of 
control  group  levels  in  Years  1 and  2,  respectively  (not  shown). 

The  increases  in  public  assistance  receipt  may  be  due  to  the  ERA  staffs  helping  pro- 
gram group  members  navigate  the  application  process.  Because  program  staff  could  work  with 
clients  after  they  became  employed,  they  may  have  been  able  to  connect  clients  to  government 
benefits  more  easily  if  they  became  eligible  (for  example,  through  job  loss).  Control  group 
members  would  not  have  had  access  to  this  postemployment  help. 

Economic  Impact  Findings,  by  Study  Cohort 

Despite  the  implementation  finding  that  the  postemployment  component  of  the  Salem 
ERA  program  was  most  fully  implemented  late  in  the  evaluation  period,  there  are  no  statistical- 
ly significant  differences  in  economic  impacts  when  analyzed  by  study  cohort. 
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Salem  ERA  Program:  Summary  and  Conclusions 

The  Salem  ERA  program  was  designed  to  be  different  from  standard  welfare-to-work 
programs  in  its  emphasis  on  helping  clients  identify  and  work  toward  a job  in  their  field  of 
interest  as  well  as  in  the  availability  of  assistance  and  services  after  employment  was  secured. 
Therefore,  the  Salem  ERA  program  included  mandatory  preemployment  services  and  offered 
voluntary  postemployment  services  to  TANF  applicants,  with  the  services  being  jointly  ad- 
ministered by  the  welfare  agency  and  a community  college.  Job  search  and  placement  services 
included  workshops  focusing  on  future  employment  retention  and  career  paths.  Then,  once 
clients  secured  employment,  the  Salem  ERA  program  intended  to  continue  with  postemploy- 
ment services  to  promote  employment  retention  and  career  advancement.  The  control  group’s 
services  were  limited  to  mandatory  preemployment  services,  also  provided  by  the  welfare  and 
the  community  college  staffs. 

While  the  Salem  ERA  program  was  designed  to  provide  both  pre-  and  postemployment 
services,  ERA  staff  had  a difficult  time  managing  both  employed  and  unemployed  clients,  and 
the  bulk  of  staff  time  was  spent  on  preemployment  responsibilities,  particularly  eligibility  issues 
and  individual  client  crises.  The  combination  of  increasing  caseloads  and  a hiring  freeze 
resulted  in  high  caseloads  for  staff,  and  the  postemployment  services  that  were  intended  to 
promote  employment  retention  and  engagement  in  training  were  not  fully  implemented.  During 
the  preemployment  period,  however,  the  Salem  ERA  program  delivered  additional  workshops 
and  other  services  that  had  a strong  retention  and  advancement  message,  compared  with 
services  for  the  control  group. 

Over  four- fifths  of  program  group  members  had  contact  with  ERA  staff  or  staff  from  an 
employment  program,  and  nearly  as  high  a proportion  participated  in  group  job  search  within  a 
year  following  study  entry.  While  participation  in  postemployment  services  was  not  as  high  — 
reflecting  the  voluntary  nature  of  those  services  — about  one-third  of  program  group  members 
received  help  with  retention  or  advancement  during  that  same  time  period. 

The  Salem  ERA  program  did  increase  the  levels  of  participation  in  some  services,  rela- 
tive to  the  control  group  levels,  but  the  increases  were  generally  less  than  10  percentage  points. 
The  one  exception  is  that  the  program  increased  the  percentage  of  program  group  members  who 
reported  that  they  received  help  dealing  with  problems  on  the  job,  by  almost  1 1 percentage 
points  over  the  control  group  rate. 

The  Salem  ERA  program  did  not  lead  to  any  changes  in  the  main  measures  of  employ- 
ment, employment  retention,  earnings,  or  advancement  in  the  cumulative  three-year  follow-up 
period.  Staff  had  difficulty  delivering  retention  and  advancement  services,  particularly  given 
their  other  job  responsibilities,  caseload  sizes,  and  lack  of  experience  in  providing  this  type  of 
service.  The  challenging  external  environment  may  have  also  contributed  to  operational 
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difficulties.  A weak  economy,  state  staffing  cutbacks,  staff  turnover,  a hiring  freeze,  and 
management  staff  whose  time  was  divided  among  a number  of  competing  duties  impacted  the 
program  throughout  the  evaluation.  The  hiring  freeze  affected  both  clients  and  staff,  in  that 
clients  were  unable  to  find  jobs  in  the  public  sector  — a major  employer  of  former  TANF 
recipients  in  Salem  — and  staff  were  overextended  as  program  positions  went  unfilled.  Overall, 
a combination  of  factors  made  it  difficult  for  ERA  staff  to  implement  this  program  and  difficult 
for  the  program  to  produce  large  increases  in  participation  in  services,  relative  to  the  control 
group.  Even  under  ideal  circumstances,  however,  the  Salem  model  may  have  been  difficult  to 
implement. 
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Chapter  4 


Findings  for  ERA  Programs  Serving 
Employed  TANF  Recipients 


Introduction 

The  Employment  Retention  and  Advancement  (ERA)  programs  that  are  presented  in 
this  chapter  were  designed  to  begin  providing  services  to  people  when  they  were  employed  and 
receiving  Temporary  Assistance  for  Needy  Families  (TANF)  and  to  help  them  retain  employ- 
ment and  advance  in  the  labor  market.  Four  programs  are  included:  the  Chicago  ERA  program, 
the  Los  Angeles  Reach  for  Success  (RFS)  program,  the  Riverside  (California)  Work  Plus 
program,  and  the  Riverside  Training  Focused  program.1  Among  these  programs,  only  one 
produced  gains  in  the  main  economic  outcomes  examined  in  this  report:  the  Chicago  ERA 
program  increased  employment  retention,  led  to  advancement,  and  reduced  welfare  receipt. 


Overview  of  Program  Designs  in  This  Group 

All  the  programs  in  this  chapter  prioritized  advancement  as  a key  program  goal  and,  by 
definition,  all  targeted  employed  TANF  recipients.  The  programs  differed  somewhat,  however, 
in  terms  of  how  long,  prior  to  program  entry,  target  groups  had  been  attached  to  the  labor 
market. 


With  shared  goals  and  target  populations,  some  common  strategies  and  structures  ex- 
isted across  these  four  programs.  Table  4.1  compares  a selection  of  features  of  the  ERA  pro- 
grams that  are  aimed  at  this  target  group  — as  they  were  actually  implemented  — and  demon- 
strates that  each  program  consisted  of  a bundle  of  features.  Appendix  B briefly  describes  each 
program  (Appendix  Boxes  B.l,  B.2  and  B.3)  and  provides  details  on  program  administration 
(such  as  operation  dates  and  funding).  A fuller  accounting  of  the  model  designs  and  implemen- 
tation of  the  programs  is  provided  in  the  ERA  site-specific  interim  reports.2 

Ongoing  relationships  between  participants  and  program  staff  were  central  to  all  the 
programs  described  in  this  chapter.  Within  that  context,  program  staff  often  provided  flexibility 
to  program  group  members,  in  terms  of  meeting  locations  and  times,  to  promote  more  frequent 
contact.  Outreach  and  marketing  were  emphasized  in  all  four  programs;  however,  in  only  one 

'The  Riverside  Work  Plus  and  Training  Focused  programs,  together,  are  referred  to  as  the  Riverside 
“Phase  2”  programs. 

2For  complete  citations,  see  Appendix  E,  which  also  notes  where  the  reports  can  be  found  on  the  Web. 
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The  Employment  Retention  and  Advancement  Project 
Table  4.1 

Comparison  of  Program  Features  Across  ERA  Programs  Targeting  Individuals 

Employed  and  Receiving  TANF 


Program  F eature 

$ 

X 

& 

V 

/ 

$ f i 

J3  & 

f i i 

c .F  F 

Staffing 

Staff  positions  - team-based  (vs.  solo) 

Staff  performance  management  (standards;  incentives) 

Flexible  staff  hours  and/or  locations 

Caseloads  - low 

Staff/client  engagement 

Participation  requirements 

TANF  participation  requirements 

ERA  participation  requirements  beyond  TANF  ones 

V 

Initial  outreach  - intensive 

V 

Initial  outreach  - financial  participation  incentives 

V 

Program  flow  - client  assessment 

and  individualized  service  package 

V 

Ongoing  - develop/maintain  an  employment  plan 

V 

Initial  job  preparation  and  placement  services 

Retention  services 

Reemployment  services 

V 

Financial  work  incentives 

V 

Emergency  financial  assistance 

Connections  to  other  services  (work  supports,  social  services) 

Staff  counseling  on  job-related  issues 

Advancement  services 

Supported  advancement  through  job  change 

V 

Education  and  training  referrals  and/or  incentives 

V 

Staff  counseling  on  job-related  issues 

Employer  linkages 

V 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E.  (For  Riverside  Phase  2 and  Chicago  ERA 
programs,  reports  were  augmented  by  communication  with  MDRC  field  researchers.) 

NOTES:  The  program  features  are  defined  in  Chapter  2. 

Check  marks  indicate  that  the  feature  is  present  in  the  ERA  model. 

The  models  and  their  implementation  often  evolved  over  the  study  period.  This  table  presents  the 
features  experienced  by  the  majority  of  the  study  participants  for  the  greatest  extent  of  time. 
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program  (Chicago  ERA)  was  participation  in  ERA-specific  activities  required  (in  addition  to 
general  TANF  participation  requirements)  for  individuals  to  remain  eligible  for  full  TANF 
benefits.  Staff  often  conducted  in-depth  assessments  of  participants,  which  largely  shaped  the 
individualized  services  for  program  group  members  going  forward.  Advancement  services 
dominated  in  all  four  programs  in  this  group  and  included  career  and  job  development  activities 
supporting  advancement  through  job  mobility,  education  and  training  referrals  and/or  incen- 
tives, and  staff  counseling  on  job-related  issues.  The  emphasis  placed  on  different  types  of 
services  varied  across  the  programs,  however.  For  example,  some  programs  prioritized  ad- 
vancement through  work,  and  others  emphasized  education  and  training  for  skill  development. 

In  addition  to  the  programs’  having  common  strategies  and  structures,  the  clients  who 
were  served  by  the  programs  described  in  this  chapter  also  shared  some  characteristics.  Table 
4.2  presents  selected  characteristics  of  the  sample  members  at  baseline,  or  the  time  of  random 
assignment.  Among  sample  members,  long-term  receipt  of  pub  lie  assistance  was  quite  common 
— more  than  half  had  received  public  assistance  for  considerable  periods  of  time,  often  as  much 
as  or  more  than  two  years,  before  they  entered  the  study.  In  addition,  roughly  half  the  sample 
members  had  at  least  a high  school  diploma  or  a General  Educational  Development  (GED) 
certificate.  Finally,  Table  4.2  also  shows  individuals’  employment  as  of  study  entry.  The  chief 
reason  that  employment  is  not  at  100  percent  for  sample  members  in  all  four  ERA  program  tests 
in  this  chapter  is  because  the  employment  measures  are  based  on  data  from  state  unemployment 
insurance  (UI)  systems.  Thus,  jobs  not  covered  by  the  UI  systems  are  not  accounted  for  in  this 
table.  As  discussed  in  Chapter  2,  UI  coverage  varies  among  states  but  generally  excludes  most 
federal,  railroad,  and  agriculture  employees  as  well  as  family  workers,  domestics,  and  indepen- 
dent contractors.  While  not  accounted  for  in  the  UI  system,  employment  in  such  non-UI- 
covered  jobs  did  qualify  people  for  inclusion  in  the  ERA  study.3 

The  clients  served  by  the  four  programs  described  in  this  chapter  do  differ  in  an  impor- 
tant way  in  terms  of  the  types  of  jobs  that  they  held  as  of  random  assignment.  In  the  Los 
Angeles  RFS  and  Riverside  Phase  2 programs,  most  people  worked  in  jobs  covered  by  the  UI 
system;  jobs  in  Chicago,  however,  were  far  less  likely  to  be  recorded  in  UI  records.  Preliminary 
analysis  conducted  during  the  Chicago  program’s  design  phase  indicated  that  many  of  the 
TANF  recipients  who  were  reporting  full-time  work  were  employed  for  less  than  the  minimum 
wage  in  cash-paying  jobs  outside  the  formal  labor  market  — for  example,  working  as  babysit- 
ters or  housecleaners. 


3The  ditference  between  client-reported  employment  and  employment  recorded  in  UI  systems  can  be  siz- 
able. For  example,  results  from  the  Chicago  ERA  12-month  survey  show  that  approximately  84  percent  of 
sample  members  reported  having  worked  in  the  year  since  study  entry.  This  exceeds  the  estimate  provided  by 
UI  records  over  roughly  the  same  time  period,  by  about  15  percentage  points. 
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Field  research  also  suggested  that  sample  members  tended  to  quickly  leave  (either  vo- 
luntarily or  involuntarily)  the  job  that  they  held  as  of  random  assignment.  This  was  true  for  all 
the  programs  described  in  this  and  other  chapters.  By  the  third  quarter  after  random  assignment, 
half  the  control  group  members  in  the  tests  described  in  this  chapter  had  left  or  lost  their  first 
jobs.4  To  the  extent  that  this  job  loss  was  involuntary,  sample  members  would  have  been  in 
need  of  reemployment  help,  in  addition  to  assistance  with  advancement. 

The  programs  in  this  chapter  also  experienced  common  implementation  challenges  in 
working  with  employed  TANF  recipients.  In  the  face  of  these  challenges,  sites  were  not  always 
able  to  implement  the  programs  as  they  were  designed;  the  following  sections  discuss  how  the 
programs  actually  operated. 


4This  refers  to  the  job  held  in  the  quarter  following  random  assignment.  This  job  either  could  have  been 
secured  before  random  assignment  and  continued  into  that  quarter  or  could  be  a newly  secured  job. 
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Chicago  ERA  Findings 


Chicago  ERA  at  a Glance 

• Targeted  long-term  TANF  recipients  in  low-wage  jobs 

• Emphasized  work-focused  advancement  strategies,  such  as  finding  better  jobs,  particu- 
larly while  people  were  working 

• Was  operated  by  a for-profit  company 

• Was  generally  well  implemented 

• Compared  with  a control  group  that  participated  in  a standard  welfare-to-work  program, 
increased  program  group  members’  contact  with  staff  and  their  use  of  retention  and  ad- 
vancement services  (particularly,  help  in  finding  a better  job  while  working) 

• Increased  employment  stability,  led  to  advancement,  and  reduced  welfare  receipt 


Chicago  ERA  Test:  Introduction 

The  Chicago  ERA  program,  which  operated  from  February  2002  to  June  2004,  aimed 
to  advance  employed  TANF  recipients  into  higher-paying  jobs.  The  study  enrolled  sample 
members  from  February  2002  through  June  2003.  The  program  served  sample  members  for  up 
to  12  months  after  their  date  of  random  assignment,  even  if  they  left  welfare. 

Origins  of  the  Test 

When  ERA  began,  Illinois  had  existing  policies  supporting  TANF  recipients  who  were 
working  in  low-paying  jobs.  A fairly  large  amount  of  recipients’  earnings  was  disregarded  when 
detennining  the  amount  of  their  TANF  cash  assistance  grant  (meaning  that  individuals  could 
maintain  a large  portion  of  their  welfare  grant  even  while  working).  Also,  working  welfare 
recipients  were  exempt  from  TANF  time  limits  (that  is,  their  five-year  TANF  “clock  stopped”) 
as  long  as  they  were  working.  Together,  these  policies  encouraged  employment  as  well  as 
ongoing  TANF  receipt.  A growing  number  of  TANF  recipients  were  in  this  “stop-the-clock” 
status  for  long  periods.  Therefore,  the  Illinois  Department  of  Human  Services  (DHS)  sought  a 
strategy  to  help  employed  recipients  advance  into  higher-paying  jobs,  both  to  improve  the 
clients’  quality  of  life  and  to  further  reduce  the  state’s  TANF  caseload.  (The  Illinois  caseload 
dropped  by  75  percent  between  1 996  and  200 1 .) 

Before  ERA,  postemployment  services  for  recipients  of  cash  assistance  were  provided 
by  local  DHS  offices  in  Chicago  (and  eventually  these  were  the  services  that  were  available  to 
control  group  members  in  the  ERA  study).  DHS  staff  provided  some  encouragement  or  assis- 
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tance  for  advancement,  but  the  intensity  and  nature  of  services  varied  substantially  across  the 
welfare  offices,  and,  on  the  whole,  DHS  staff  communicated  with  participants  primarily  about 
compliance  issues.5 

In  contrast,  the  Chicago  ERA  program  was  a mandatory  and  more  comprehensive 
program,  emphasizing  work-focused  retention  and  advancement.  The  program  included  a 
combination  of  services  provided  through  ongoing  staff-client  relationships,  including 
targeted  job  search  assistance  and  help  identifying  and  accessing  employers  and  jobs  with 
opportunities  for  progression  from  entry-level  positions  to  those  with  higher  levels  of  pay, 
skill,  responsibility,  or  authority.6  The  Chicago  ERA  program  was  operated  by  a for-profit 
company  with  strong  linkages  to  firms  in  a variety  of  industries,  including  the  fast-growing 
security  industry.  Moreover,  this  provider  had  previously  played  an  employment  intermediary 
role  and  had  capitalized  on  its  relationships  with  local  employers  to  identify  open  jobs  and 
facilitate  targeted  job  search  assistance  for  welfare  recipients. 

Labor  Market  Context 

For  most  of  the  period  when  the  Chicago  ERA  program  was  operating,  the  area’s  labor 
market  was  still  feeling  the  effects  of  the  2001  recession;  between  2001  and  2003,  the  unem- 
ployment rate  for  the  Chicago  area  increased  from  4.3  percent  to  6.8  percent,  and  its  total 
number  of  jobs  declined  by  over  170,000.  While  labor  market  conditions  began  to  improve  in 
2004,  overall  job  growth  was  fairly  sluggish.  As  of  2007,  the  unemployment  rate  and  the  total 
jobs  in  the  area  were  still  above  prerecession  levels.  While  the  area’s  economic  situation  was 
difficult,  the  situation  in  Chicago  itself  (part  of  Cook  County)  was  even  more  challenging.  In 
particular,  most  of  the  jobs  lost  during  the  postrecession  slump  were  in  Cook  County,  but  most 
of  the  job  growth  after  2004  in  the  Chicago  area  was  in  establishments  located  outside  Cook 
County.  So  even  though  labor  market  conditions  in  the  Chicago  area  did  improve  toward  the 
end  of  the  Chicago  ERA  program’s  operating  period  and  throughout  the  study’s  follow-up 
period,  conditions  in  sample  members’  immediate  vicinity  were  (and  remained)  far  more 
challenging. 


5Working  individuals  who  were  receiving  TANF  were  not  required  to  participate  in  advancement- 
promoting  services,  but  they  were  required  to  work  or  participate  in  services  for  a certain  number  of  hours  each 
week.  In  the  event  of  job  loss  or  reduced  work  hours,  individuals  were  required  to  participate  in  services 
designed  to  help  them  find  a new  job  (or  increase  their  hours),  provided  either  by  DHS  staff  directly  or  by  a 
contracted  employment  vendor. 

6Such  employment  opportunities  are  later  referred  to  as  “career  ladders.” 
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Target  Population 

The  Chicago  ERA  program  targeted  working  TANF  recipients  who  reported  at  least  30 
hours  per  week  of  work  for  at  least  six  consecutive  months  and  who  were  served  by  select 
welfare  offices  in  Cook  County.  Table  4.2  shows  the  characteristics  of  the  Chicago  ERA  sample 
members  at  the  time  the  study  began.  The  Chicago  sample  had  the  highest  average  age  among 
the  ERA  samples,  at  33  years,  and  an  above-average  number  of  children,  compared  with  the 
other  ERA  samples.  This  pattern  is  likely  explained  by  the  fact  that  recipients  with  larger 
families  qualify  for  larger  grant  amounts  and,  therefore,  are  able  to  earn  more  and  still  maintain 
their  eligibility  for  TANF  benefits.  Finally,  unique  to  the  Chicago  ERA  sample  was  the  large 
proportion  of  sample  members  who  were  working  outside  the  formal  labor  market,  in  low- 
paying  jobs  such  as  babysitting,  housecleaning,  and  hair  braiding. 

Significance  of  This  Test 

The  Chicago  ERA  test  examines  whether  a work-focused,  mandatory  advancement 
program  — provided  by  staff  in  a for-profit  employer-intermediary  — can  move  employed 
TANF  recipients  into  better  or  higher-paying  jobs,  compared  with  a less  intensive  and  more 
retention-oriented  postemployment  program  provided  by  staff  at  the  local  welfare  agency.  Chart 
4.1  presents  the  important  differences  between  the  situations  for  the  program  group  and  the 
control  group  in  the  Chicago  ERA  test. 

Chicago  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

DHS  developed  Chicago  ERA,  and  it  was  funded  with  resources  from  the  state’s  Wel- 
fare-to-Work  block  grant.  Considerable  funding  was  available  for  the  program  initially.  Howev- 
er, due  to  a congressional  decision  in  early  2004  to  rescind  unspent  funds  in  the  block  grant,  the 
program  experienced  a brief  gap  in  funding.  The  rescission  did  not  officially  occur  until  January 
2004;  however,  it  was  widely  discussed  beginning  in  the  fall  of  2003.  From  that  point  forward, 
according  to  field  research,  the  substantial  uncertainty  about  the  ERA  program’s  future  affected 
staff  morale,  and  program  services  began  to  phase  out.  As  a result,  individuals  who  were 
randomly  assigned  to  the  program  group  in  the  last  few  months  (April  to  June  2003)  of  the 
sample  intake  period  received  more  limited  exposure  to  the  program  than  those  who  were 
assigned  earlier. 

For  administration  of  ERA  services,  DHS  contracted  with  Employment  and  Employer 
Services  (E&ES),  a for-profit  firm  with  previous  experience  delivering  job  placement  services 
to  TANF  recipients  and  other  disadvantaged  populations.  Since  its  establishment  in  1982, 
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The  Employment  Retention  and  Advancement  Project 
Table  4.2 


Selected  Characteristics  of  ERA  Sample  Members  at  the  Time  of  Random  Assignment: 

Employed  and  Receiving  TANF 


Characteristic2 

Chicago 

Los  Angeles 
RFS 

Riverside 
Phase  2 

Female  (%) 

99.3 

94.1 

92.2 

Average  age  (years) 

33.3 

30.8 

30.2 

Race/ethnicity  (%) 

Hispanic 

8.0 

40.7 

45.4 

Black,  non-Hispanic 

87.5 

50.5 

20.5 

White,  non-Hispanic 

3.9 

5.2 

31.2 

Other 

0.6 

3.6 

3.0 

Average  number  of  minor  children 

3.4 

2.3 

2.2 

Age  of  youngest  child  (%) 

2 or  under 

26.0 

40.3 

46.4 

3 to  5 

23.3 

25.1 

22.0 

6 or  over 

50.7 

34.6 

31.7 

High  school  diploma/GED  certificate  or  higher  (%) 

44.2 

50.0 

58.2 

Total  prior  AFDC/TANF  receiptb(%) 

None 

NA 

9.9 

5.5 

Less  than  2 years 

NA 

39.4 

53.5 

2 years  or  more 

NA 

50.8 

41.0 

Living  in  public  or  subsidized  housing  (%) 

NA 

12.0 

12.6 

Employed  in  quarter  prior  to  random  assignmentc(%) 

58.7 

46.1 

57.2 

Employed  in  quarter  of  random  assignment°(%) 

59.5 

78.9 

87.2 

Employment  in  the  past  3 years  (%) 

Did  not  work 

NA 

NA 

5.6 

Less  than  6 months 

NA 

NA 

25.2 

7 to  12  months 

NA 

NA 

19.8 

13  to  24  months 

NA 

NA 

22.5 

More  than  24  months 

NA 

NA 

26.9 

As  of  random  assignment  the  number  of 
hours  worked  per  week  among  employed  (%) 

1-19  hours 

0.6 

0.4 

0.9 

20-31  hours 

50.6 

1.0 

43.4 

32  or  more  hours 

48.8 

98.6 

55.7 

As  of  random  assignment  the  hourly  wage 
among  employed  (%) 

Less  than  $5.15 

NA 

0.1 

0.0 

$5.15  to  $6.99 

NA 

31.2 

48.5 

$7.00  to  $9.99 

NA 

55.3 

43.6 

$10.00  or  more 

NA 

13.4 

7.9 

Sample  size 

1,728 

5,700 

3,029 

(continued) 
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Table  4.2  (continued) 


SOURCES:  Calculations  from  ERA  baseline  forms,  automated  records,  and  administrative  data. 

NOTES:  NA  = not  available.  In  this  case,  the  data  for  these  measures  were  not  collected. 

“Statistics  include  both  program  and  control  group  members. 

bIn  constructing  the  total  prior  welfare  receipt  variable  in  the  Riverside  Training  Focused  and  Riverside 
Work  Plus  sites,  the  measure  is  based  on  estimates  of  how  much  TANF  sample  members  have  received  in  the 
previous  10  years. 

“This  information  is  based  on  unemployment  insurance  (UI)  records. 


E&ES  had  developed  strong  connections  with  many  local  employers,  particularly  within  the 
security  industry. 

Staffing 

The  Chicago  ERA  program  had  a relatively  simple  staffing  structure,  based  largely  on  a 
single,  “generic”  staff  position.  Staff  were  responsible  for  all  aspects  of  the  ERA  program, 
including  the  development  of  a client-specific  employment  plan  and  the  identification  of  job 
openings  with  local  employers.  For  most  of  the  study  period,  there  were  seven  to  eight  ERA 
staff  working  individually  with  participants,  rather  than  in  a team  structure.  ERA  caseloads 
were  small  at  the  outset  of  the  program  but  grew  over  time.  Additional  specialized  staff  served 
program  group  members’  job  development  needs  and  provided  workshops  focusing  on  em- 
ployability skills,  computer  training,  and  other  topics. 

The  structure  of  the  Chicago  ERA  program  did  not  provide  much  flexibility  for  pro- 
gram group  members  in  terms  of  meeting  locations.  Although  E&ES  operated  programs  in 
several  locations,  for  most  of  the  study  period,  all  the  ERA  staff  were  based  in  the  organiza- 
tion’s main  office  in  downtown  Chicago.  Each  member  of  the  ERA  staff  was  assigned  to  work 
with  participants  from  one  or  two  of  the  ten  participating  DHS  offices,  which  were  dispersed 
around  the  city.  Typically,  ERA  staff  did  not  conduct  home  visits  and  only  rarely  visited 
participants  at  their  workplaces.  Thus,  program  group  members  generally  had  to  travel  down- 
town for  face-to-face  meetings  with  ERA  staff. 

The  ERA  staff  were  given  specific  quarterly  performance  goals  related  to  the  participation 
of  program  group  members  in  education  and  training  activities  or  to  earnings  increases,  and  staff 
could  earn  financial  bonuses  for  meeting  or  exceeding  these  targets.7  ERA  staff  members 


7tnitially,  the  targets  were  defined  narrowly:  staff  needed  to  raise  participants’  hourly  wages  or  increase 
their  work  hours  to  generate  at  least  a 6 percent  increase  in  gross  earnings.  Because  of  concerns  that  these  goals 
did  not  fully  reward  the  range  of  advancement-related  services  that  staff  provided,  in  early  2003,  the  perfor- 

(continued) 
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The  Employment  Retention  and  Advancement  Project 
Chart  4.1 

Program-Control  Group  Differences: 
Chicago  ERA  Test 


ERA  Group 

Control  Group 

Goals 

• Advance  employed  TANF  recipients  into 
higher-paying  jobs 

• Primarily  retention;  compliance  with 
TANF  work  requirements 

Program 

• Operated  by  for-profit  company 

• Operated  by  public  welfare  agency 

resources  and 
structures 

Staffing 

• Staff  performance  incentives  (financial) 

• No  staff  performance  incentives 

Staff-client 

engagement 

• ERA-service  participation  requirements, 
beyond  TANF  participation  requirements 

• Aggressive  and  multifaceted  marketing  and 
outreach,  including  financial  participation 
incentives 

• Services  tailored  to  participants 

• Development  and  maintenance  of  client 
employment  plan 

• Follow-up  after  leaving  TANF 

• Only  TANF  participation  requirements 

• Primarily  compliance-related  outreach 

• Generic  service  package 

• No  employment  plan 

• No  follow-up  after  leaving  TANF 

Retention 

• Client  employment  incentives  (financial) 

• No  client  employment  incentives 

services 

Advancement 

• Services  for  advancement  through  job  change 

• No  advancement  services 

services 

• Limited  education  and  training 
referrals/incentives 

• No  education  and  training  referrals/ 
incentives,  but  achieved  similar  take-up  rates 
as  ERA 

Employer 

linkages 

• Strong  relationships  with  local  employers 

• Staff  identification  of  jobs  with  local 
employers 

• No  links  to  employers 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


mance  measurement  system  was  broadened  to  including  enrolling  in  education  or  training,  opening  a bank 
account,  or  obtaining  the  Earned  Income  Tax  Credit  (EITC). 
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also  received  internal  training,  as  well  as  training  from  a consultant  retained  by  MDRC  (as  in 
many  of  the  other  ERA  programs).  The  MDRC-sponsored  training  focused  on  how  to  market 
ERA  services  to  potential  participants,  how  to  develop  employment  plans  for  individuals,  and 
how  to  document  the  staffs  work  with  program  group  members. 

Staff-Client  Engagement 

Ongoing  staff-client  relationships  were  central  to  the  Chicago  ERA  program.  A combi- 
nation of  incentives  and  mandates  was  used  to  contact  program  group  members  initially  and  to 
encourage  their  ongoing  participation.  The  ERA  program  required  clients  to  have  regular 
contact  with  staff  in  addition  to  their  existing  30-hour  work  requirement  for  ongoing  TANF 
eligibility.  Failure  to  participate  could  result  in  a sanction,  meaning  that  their  welfare  grant 
could  be  reduced  or  ended.8  However,  because  these  working  clients  were  receiving  relatively 
small  welfare  checks,  ERA  staff  reported  that  some  program  group  members  chose  to  leave 
cash  assistance  in  order  to  avoid  the  requirement  to  participate  in  ERA.  Thus,  while  the  TANF 
enforcement  process  and  the  Chicago  ERA  participation  requirement  may  have  been  useful  in 
the  initial  recruitment  process,  it  may  have  played  less  of  a role  in  promoting  ongoing  engage- 
ment with  the  ERA  program. 

At  the  same  time,  Chicago  ERA  staff  used  aggressive  marketing  and  recruitment  strate- 
gies — including  a dedicated  telemarketer,  a range  of  flyers,  reminder  letters,  and  newsletters 
— as  part  of  their  initial  outreach.  Financial  incentives  were  also  used  to  attract  and  engage 
ERA  members.  These  incentives  were  offered  for  different  types  of  activities.  McDonald’s  gift 
certificates  of  $10,  for  example,  accompanied  the  introductory  letter  to  the  ERA  program  group; 
$50  grocery-store  gift  certificates  were  given  to  those  who  attended  the  orientation  and  assess- 
ment; $25  to  $125  one-time  rewards  in  the  form  of  grocery-store  gift  certificates  were  given  to 
individuals  who  achieved  specific  milestones,  such  as  completing  a vocational  training  pro- 
gram, starting  a better  job,  or  retaining  a better  job  for  90  days;  and  monthly  transit  passes  worth 
$75  were  offered  to  employed  recipients  who  remained  in  contact  with  the  program.  According 
to  field  research,  the  ERA  staff  faced  challenges  engaging  this  employed  population  — a 
difficulty  experienced  by  all  the  ERA  programs  serving  this  group.  Staff  had  difficulty  convinc- 
ing individuals  of  the  benefits  of  advancement.  Staff  reported  that  some  clients  resisted  formal 
employment,  which  was  often  in  the  less  familiar  downtown  Chicago  neighborhoods  and 
offered  relatively  small  wage  increases  compared  with  the  informal  jobs  they  held.  Field 


8While  ERA  staff  were  initially  hesitant  to  penalize  noncompliance,  field  research  suggests  that  sanctions 
were  used  increasingly  over  time.  In  Illinois,  an  imposed  sanction  can  result  in  eventual  forfeiture  of  an 
individual’s  full  TANF  grant,  with  reductions  in  the  grant  amount  occurring  initially.  In  some  other  states,  an 
imposed  sanction  results  in  only  a reduction  in  the  amount  of  an  individual’s  TANF  grant. 
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research  also  suggested  that  many  people  in  the  program  group  were  not  interested  in  receiving 
program  services  or  faced  serious  personal  or  family  problems  that  hindered  their  participation. 

Despite  these  challenges,  more  than  half  (61  percent)  of  the  program  group  members 
reported  having  had  contact  with  ERA  staff  or  staff  from  an  employment  program,  when 
surveyed  12  months  following  study  entry.  As  shown  in  Table  4.3,  this  rate  of  contact  is  nearly 
twice  the  rate  reported  by  control  group  members. 

Chicago  ERA  staff  had  discretion  in  the  types  of  services  they  provided.  Expected  to 
tailor  services  to  individuals,  staff  conducted  informal  client  assessments  during  initial  meet- 
ings, which  provided  the  foundation  for  the  ongoing  staff-client  relationship.  In  particular,  ERA 
staff  developed  an  employment  plan  with  their  clients,  identifying  short-  and  long-tenn  goals, 
barriers  to  achieving  those  goals,  and  steps  and  services  for  overcoming  the  barriers.  The  staff- 
client  relationship  in  the  Chicago  ERA  program  often  continued  after  people  left  TANF. 

Survey  data  suggest  that  the  Chicago  ERA  program  increased  the  number  of  contacts 
that  program  group  members  — compared  with  control  group  members  — had  with  staff  from 
an  employment  program  as  well  as  the  likelihood  that  contact  would  be  ongoing.  ERA  program 
group  members  reported  having  three  times  the  number  of  contacts,  on  average,  with  staff  over 
the  first  12  months  after  study  entry  (Table  4.3).  Similarly,  more  than  two  times  as  many 
program  group  members  as  control  group  members  reported  having  had  recent  contact. 

Initial  Job  Preparation  and  Placement  Services;  Retention  Services 

The  Chicago  ERA  program  targeted  TANF  recipients  who  had  six  consecutive  months 
of  full-time  work;  therefore,  program  designers  initially  assumed  that  this  population  would  not 
need  immediate  job  placement  services.  Thus,  neither  initial  job  preparation  and  placement 
services  nor  retention  services  were  priorities  of  the  Chicago  ERA  program. 

However,  a substantial  number  of  program  group  members  in  Chicago  were,  in  fact,  no 
longer  employed  when  they  first  had  contact  with  program  staff,  and  staff  provided  reemploy- 
ment services  to  them.  The  time  lag  between  random  assignment  and  first  program  contact  may 
partially  explain  this  level  of  unemployment.  Initial  ERA  appointments  often  were  not  sched- 
uled until  six  to  eight  weeks  after  random  assignment.  According  to  the  12-month  survey, 
nearly  40  percent  of  respondents  in  the  program  group  received  help  with  job  preparation  — 
more  than  twice  the  proportion  of  respondents  in  the  control  group  who  reported  receiving  this 
service  (Appendix  Table  B.l). 
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The  Employment  Retention  and  Advancement  Project 
Table  4.3 


Summary  of  Impacts: 
Chicago 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

61.0 

31.3 

29  g *** 

0.000 

Average  number  of  contacts  with  staff 

9.8 

2.8 

6.9  *** 

0.000 

Talked  with  staff  in  past  4 weeks  (%) 

25.9 

9.6 

16.3  *** 

0.000 

Retention  services 

Participated  in  a job  search  activity  (%) 

56.5 

35.7 

2Q  7 *** 

0.000 

Received  help  dealing  with  problems  on  the  job  (%) 

8.7 

4.5 

4 2 ** 

0.043 

Advancement  services 

Received  help  with  career  assessment  (%) 

21.7 

6.7 

1 *** 

0.000 

Received  help  finding  a better  job  while  working  (%) 

28.0 

5.2 

22.8  *** 

0.000 

Participated  in  an  education/training  activity3  (%) 

23.3 

24.9 

-1.6 

0.648 

Received  any  help  with  retention/advancement"  (%) 

37.0 

12.6 

24  5 *** 

0.000 

Sample  size  (total  = 598) 

306 

292 

Cumulative  economic  outcomes  (Years  1-4) 

Employment  measures 

Ever  employed  (%) 

80.5 

79.2 

1.3 

0.434 

Average  quarterly  employment  (%) 

55.0 

52.4 

2.6  * 

0.069 

Had  employment  spell  of  at  least  4 quarters  (%) 

64.8 

62.9 

1.8 

0.342 

Length  of  longest  employment  spell,  in  quarters 

7.7 

7.3 

0.4  * 

0.091 

Length  of  longest  unemployment  spell,  in  quarters 

6.5 

6.9 

-0.4  * 

0.054 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

6,967 

6,490 

477  * 

0.074 

Quarters  with  earnings  of  $3,500  or  more  (%) 

21.6 

19.0 

2.6  ** 

0.035 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

42.4 

46.6 

-4.2  ** 

0.027 

Earnings  decreased  by  more  than  $250 

13.9 

12.8 

1.1 

0.485 

Earnings  changed  by  less  than  $250 

6.4 

7.1 

-0.7 

0.570 

Earnings  increased  by  $250  or  more 

37.2 

33.6 

3.5  * 

0.095 

Public  assistance  and  income  (Years  1-3) 

Average  annual  TANF  received  ($) 

778 

1,010 

-232  *** 

0.000 

Average  annual  food  stamps  received  ($) 

4,111 

3,959 

153  ** 

0.049 

Average  annual  incomec  ($) 

11,467 

11,118 

349 

0.163 

Sample  size  (total  = 1,728) 

854 

874 

(continued) 
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Table  4.3  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

cThis  measure  represents  the  sum  of  Ui-reported  earnings,  TANF,  and  food  stamps. 


While  retention  services  were  not  included  in  the  original  Chicago  ERA  model,  ERA 
staff  spent  considerable  time  connecting  participants  with  such  assistance  as  social  service 
referrals  for  mental  health  and  substance  abuse  services,  and  they  provided  help  with  acquiring 
such  work  supports  as  child  care  subsidies. 

Advancement  Services 

Despite  these  unforeseen  demands  for  initial  placement  services,  advancement  services 
were  the  focus  of  the  Chicago  ERA  program  and  its  services.  The  primary  advancement 
strategy  in  the  program  was  to  help  individuals  move  fairly  quickly  to  a new  job  that  paid  more 
than  their  current  job,  preferably  in  firms  or  sectors  that  offered  access  to  career  ladders.  ERA 
staff  identified  appropriate  job  openings  — usually  among  the  employers  that  the  Chicago  ERA 
contracted  agency  had  a history  working  with  — and  helped  with  resume  writing  and  the 
scheduling  of  and  preparation  for  interviews. 

According  to  field  research,  the  ERA  service  provider  had  strong  linkages  with  firms  in 
the  fast-growing  security  industry,  and  many  program  group  members  were  referred  to  jobs  in 
those  firms.  This  focus  on  advancement  through  job  mobility  likely  led  to  the  significantly 
greater  proportion  of  program  group  members  who  reported  receiving  help  finding  a better  job 
while  working  in  the  first  year  after  random  assignment,  compared  with  control  group  members 
(more  than  five  times  the  proportion,  as  shown  in  Table  4.3). 

The  amount  of  time  spent  on  career  counseling  and  on  the  development  and  mainten- 
ance of  employment  plans  in  the  Chicago  ERA  program  was  limited  because  staff  also  had  to 
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address  sample  members’  job  search  and  retention  needs.  Perhaps  reflecting  this,  less  than  a 
quarter  of  program  group  survey  respondents  reported  receiving  help  with  career  assessment 
during  the  first  year  following  study  entry  (Table  4.3).  Nevertheless,  a greater  portion  of 
program  group  members  than  control  group  members  reported  receiving  career  assessment 
help. 


Education  and  training  services  (including  referrals  to  outside  service  providers  and  as- 
sistance with  tuition  payments)  were  included  in  the  Chicago  ERA  program.  However,  in  the 
12-month  survey,  less  than  one-quarter  of  program  group  members  reported  that  they  had 
participated  in  education  and  training  programs  — a proportion  no  different  than  in  the  control 
group.  Reflecting  the  ERA  service  provider’s  connections  to  security  industry  employers,  a 
higher  proportion  of  program  group  members  than  control  group  members  reported  that  they 
had  received  licenses  and  certifications  in  the  security  industry,  according  to  the  42-month 
survey.  This  is  the  only  statistically  significant  impact  on  education  and  training  activity  using 
the  longer-term  survey  data  (Appendix  Table  D.2). 

Employer  Linkages 

As  noted  above,  Chicago  ERA  staff  in  the  private,  for-profit  firm  that  was  contracted  to 
operate  the  program  had  strong  existing  relationships  with  local  employers.  This  enabled  staff  to 
direct  participants  to  particular  finns  and  positions.  Moreover,  ERA  staff  were  also  required  to 
identify  jobs  as  being  appropriate  for  certain  program  group  members  as  part  of  their  ERA 
responsibilities. 

Chicago  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Chicago  ERA  program  can  be  summarized  as  follows: 

• The  ERA  program  in  Chicago  generated  improvements  in  several  measures 
of  employment  retention  and  earnings  in  the  cumulative  four-year  follow-up 
period.  There  is  also  evidence  that  ERA  led  to  advancement  in  the  four-year 
follow-up  period. 

• ERA  decreased  the  amount  of  TANF  and  increased  the  amount  of  food 
stamps  received  during  the  first  three  years  of  follow-up  (though  the  increase 
in  food  stamp  assistance  was  smaller  in  magnitude  than  the  decrease  in 
TANF  assistance).  ERA  did  not  produce  any  statistically  significant  impacts 
on  average  annual  income  during  the  cumulative  three-year  follow-up  period 
in  which  TANF  and  food  stamp  data  were  available. 
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Control  Group  Outcomes 

Average  quarterly  employment  and  average  earnings  were  lower  for  control  group 
members  in  Chicago  than  for  control  group  members  in  the  other  tests  of  programs  serving 
employed  TANF  recipients.  This  may  partially  reflect  the  Chicago  sample’s  higher  rates  of 
employment  in  jobs  not  covered  by  the  UI  system,  compared  with  the  samples  studied  in  the 
other  tests  described  in  this  chapter.  Average  quarterly  employment  was  just  over  50  percent, 
and  average  earnings  were  almost  $6,500  per  year  for  control  group  members  over  the  four-year 
follow-up  period  (Table  4.3).  Figure  4.1  shows  that  average  earnings  increased  relatively 
steadily  over  the  follow-up  period  — without  a corresponding  increase  in  employment9  — 
which  may  indicate  that  there  was  some  advancement  over  time  in  the  control  group.  While 
most  of  the  control  group  members  received  at  least  one  TANF  payment  in  the  first  follow-up 
year,  in  the  third  year  of  follow-up,  only  28  percent  of  the  control  group  received  at  least  one 
TANF  payment  (Appendix  Table  B.2). 

Employment  and  Employment  Retention 

The  Chicago  ERA  program  produced  gains,  relative  to  the  control  group,  in  several 
measures  of  employment  retention  in  the  cumulative  four-year  follow-up  period  (Table  4.3). 
The  program  increased  average  quarterly  employment  by  2.6  percentage  points  in  the  four-year 
follow-up  period.  The  program  also  increased  the  average  length  of  the  longest  employment 
spell  by  0.4  quarter,  or  by  about  5 percent  of  the  control  group  average. 

The  Chicago  ERA  program’s  early  impacts  appear  to  reflect  that  the  program  moved 
people  from  jobs  not  in  the  UI  system  to  Ul-covered  jobs.  This  is  important  because  Ul-covered 
jobs  typically  pay  higher  wages  and  are  more  likely  to  offer  fringe  benefits  and  other  mandatory 
benefits,  such  as  Social  Security  and  unemployment  benefits  and  to  qualify  individuals  for  the 
Earned  Income  Tax  Credit  (EITC).  While  the  program  did  not  have  any  effect  on  the  percent- 
age ever  employed  in  Ul-covered  jobs  in  the  cumulative  four- year  follow-up  period,  it  did  move 
people  into  Ul-covered  employment  faster.  The  program  decreased  the  number  of  quarters, 
relative  to  the  control  group,  that  elapsed  before  program  group  members  started  their  first 
employment  spell  in  a Ul-covered  job  (Appendix  Table  B.3).10  In  addition,  while  the  program 
produced  increases  in  the  percentage  ever  employed  in  Year  1 (Appendix  Table  B.2),  the  12- 
month  survey  shows  that  when  all  jobs  are  considered  (including  those  not  reported  to 


9Because  people  who  are  not  employed  in  a given  quarter  have  earnings  of  zero,  and  because  these  zeros 
are  included  in  average  earnings,  average  earnings  can  increase  over  time  as  employment  increases  even  if 
earnings  did  not  increase  at  all  for  people  who  were  employed. 

10Sample  members  who  were  employed  in  a Ul-covered  job  in  the  quarter  of  study  entry  would  have  a 
value  of  zero  for  this  measure. 
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Earnings  ($)  Percentage  employed 


The  Employment  Retention  and  Advancement  Project 


Figure  4.1 

Impacts  on  Employment  and  Earnings  Over  Time: 
Chicago 


SOURCE:  MDRC  calculations  from  administrative  records  for  the  State  of  Illinois. 

NOTES:  "RA"  refers  to  the  quarter  of  random  assignment. 

Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings. 
Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  and  * = 10 
percent. 
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the  UI  system),  the  Chicago  program  did  not  increase  employment  in  Year  1 (Appendix  Table 
B.4).  Finally,  gains  in  the  first  two  years  of  follow-up  were  concentrated  among  those  who  did 
not  work  in  a Ul-covered  job  in  the  six  months  prior  to  random  assignment.1 * * * * * * * * * 11  These  findings 
strongly  suggest  that  one  of  the  program’s  primary  accomplishments  in  Year  1 was  to  move 
people  who  had  not  been  in  Ul-covered  employment  into  Ul-covered  jobs. 

There  is  also  evidence  that  the  Chicago  ERA  program  helped  people  become  reem- 
ployed more  quickly  after  a loss  of  employment,  particularly  early  on  in  program  operations.  In 
addition  to  increasing  the  length  of  employment  spells,  the  program  also  decreased  the  average 
length  of  the  longest  unemployment  spell  by  about  0.4  quarter,  or  by  about  6 percent  of  the 
control  group  average  (Table  4.3).  There  is  evidence  that  at  least  part  of  this  effect  was  due  to 
the  program’s  helping  program  group  members  become  reemployed  more  quickly  than  control 
group  members  after  loss  of  employment. 12  Most  program  and  control  group  members  left  their 
initial  jobs  (the  jobs  they  held  at  the  time  of  random  assignment)  during  the  follow-up  period, 
and  they  left  these  jobs  at  a roughly  equivalent  rate.  Thus,  there  is  no  evidence  that  the  Chicago 
ERA  program  had  an  effect  on  retention  of  this  initial  job.  At  the  same  time,  in  the  first  two 
years  of  follow-up,  the  program  increased  the  average  number  of  employers  for  program  group 
members,  compared  with  control  group  members  (not  shown).13  Given  that  program  group 
members  left  their  initial  job  at  the  same  rate  as  control  group  members  and  given  that  program 
group  members  had  more  employers  early  on  in  the  follow-up  period,  this  suggests  that  the 
Chicago  program  increased  the  proportion  of  people  who  quickly  found  a subsequent  job  or  that 
it  helped  people  switch  to  other  jobs  when  they  were  already  employed.14  In  addition,  other 
evidence  also  supports  the  finding  that  the  Chicago  ERA  program  helped  people  become 
reemployed  more  quickly  after  a loss  of  employment. 

The  Chicago  ERA  program  generated  some  gains  in  employment  in  the  first  year  of  fol- 
low-up, and  these  gains  increased  in  magnitude  in  Year  2,  growing  to  about  6 percentage  points  in 
its  third  quarter  (Figure  4.1).  Despite  the  fact  that  program  group  members  were  eligible  for 
services  only  for  one  year  following  their  entry  into  the  study,  the  program  had  its  greatest  effect 


1 'Bloom,  Hendra,  and  Page  (2006). 

12Some  of  this  decrease  also  could  have  resulted  from  the  program’s  getting  people  into  Ul-covered  em- 

ployment more  quickly  or  leading  some  people  who  would  have  had  an  unemployment  spell  without  the 

program  to  be  continuously  employed  throughout  the  follow-up  period. 

1 'Bloom,  Hendra,  and  Page  (2006),  Appendix  Table  B.  1 1 . 

14 While  the  program  did  increase  the  average  number  of  employers  in  the  first  two  years  of  follow-up,  it 

did  not  affect  the  average  number  of  employers  over  the  cumulative  four-year  follow-up  period.  However,  it 

did  appear  to  increase  the  percentage  of  program  group  members  who  had  five  to  eight  employers  over  the 

four-year  follow-up  period,  compared  with  the  percentage  seen  in  the  control  group  (Appendix  Table  B.3). 

This  could  provide  further  evidence  that  the  program  helped  people  with  unstable  jobs  regain  employment 

more  quickly  after  job  loss  or  that  it  helped  people  switch  to  better  jobs. 
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on  employment  retention  in  Year  2,  increasing  the  percentage  employed  in  every  quarter  in  Year  2 
by  4.5  percentage  points  (Appendix  Table  B.2).  Statistically  significant  impacts  on  employment 
and  employment  retention  dissipated  in  the  last  two  years  of  the  follow-up  period. 

Earnings  and  Advancement 

The  ERA  program  in  Chicago  increased  earnings  and  appears  to  have  led  to  additional 
advancement,  compared  with  the  level  for  the  control  group.  ERA  produced  gains  in  earnings 
over  the  cumulative  four-year  follow-up  period,  increasing  average  annual  earnings  by  almost 
$500,  or  by  about  7 percent  more  than  the  control  group  earnings  level  (Table  4.3).  Statistically 
significant  impacts  on  earnings  appeared  in  the  second  half  of  Year  2 and  were  also  present  in 
some  quarters  of  Years  3 and  4 (Figure  4.1).  The  earnings  increased  in  the  last  two  years  of 
follow-up,  when  there  were  no  gains  in  quarterly  employment  during  that  period,  suggesting 
that  these  gains  were  due  to  advancement  rather  than  just  being  the  result  of  the  program’s 
effects  on  employment.15  Further  evidence  of  possible  advancement  over  time  can  be  seen  by 
comparing  the  highest  earnings  quarter  in  Year  4 with  the  highest  earnings  quarter  in  Year  1. 
About  37  percent  of  program  group  members  made  at  least  $250  more  in  their  highest-paying 
quarter  in  Year  4 than  in  their  highest-paying  quarter  in  Year  1,  compared  with  only  34  percent 
of  control  group  members  (Table  4.3). 

The  survey  data  also  provide  evidence  that  the  Chicago  ERA  program  increased  ad- 
vancement, and  they  show  that  the  earnings  increases  were  at  least  partly  due  to  increases  in 
wage  rates.  As  measured  by  the  12-month  survey,  the  program  appears  to  have  shifted  some 
people  from  making  less  than  $7  per  hour  to  making  more  than  $7  per  hour  (Appendix  Table 
B.4).  Table  4.4  shows,  however,  that  the  program’s  effect  on  hourly  wages  is  less  clear  for  the 
survey  sample  by  the  time  of  the  42 -month  survey;  while  the  program  did  decrease  the  percent- 
age of  program  group  members  making  less  than  $7  per  hour,  there  is  no  statistically  significant 
increase  in  the  percentage  of  them  making  more  than  $7  per  hour. 

Despite  the  fact  that  program  group  members  were  eligible  for  services  only  for  one  year, 
the  Chicago  ERA  program  appears  to  have  increased  some  measures  of  long-term  advancement, 
compared  with  control  group  levels,  based  on  42 -month  survey  results.  The  program  increased  the 
percentage  of  program  group  members  who  had  ever  received  a raise  and  the  percentage 


15  The  Chicago  ERA  program  appears  to  have  affected  employment  retention  earlier  than  it  affected  ad- 
vancement. The  fact  that  the  program’s  main  effects  on  advancement  occurred  after  program  group  members 
would  no  longer  have  been  eligible  for  program  services  and  the  fact  that  the  program’s  effects  on  retention 
seem  to  have  preceded  its  effects  on  advancement  suggest  that  the  program  may  have  increased  advancement 
over  time  partly  through  its  early  effects  on  boosting  levels  of  employment  and  employment  retention  in  UI- 
coveredjobs. 
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The  Employment  Retention  and  Advancement  Project 
Table  4.4 


Impacts  on  Current  Job  Characteristics  and  Advancement 
at  the  42-Month  Survey: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

92.9 

89.6 

3.3  * 

0.063 

Currently  employed 

66.6 

67.7 

-1.1 

0.706 

No  longer  employed 

26.3 

21.9 

4.4 

0.105 

Hourlv  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

7.9 

9.2 

-1.3 

0.483 

$5.00 -$6.99 

12.1 

16.1 

-4.0  * 

0.081 

$7.00  - $8.99 

21.3 

20.1 

1.2 

0.654 

$9.00  or  more 

23.3 

21.4 

1.9 

0.503 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

10.7 

14.1 

-3.4 

0.106 

30-34 

12.1 

13.6 

-1.5 

0.489 

35-44 

38.0 

35.6 

2.4 

0.428 

45  or  more 

5.6 

4.5 

1.1 

0.424 

Schedule 

Typical  schedule  (%) 

Daytime  schedule 

33.9 

37.4 

-3.4 

0.252 

Evening  shift 

7.2 

7.2 

-0.1 

0.956 

Other  schedule 

23.8 

21.0 

2.8 

0.281 

Workweek  includes  at  least  1 weekend  day  (%) 

31.1 

32.2 

-1.1 

0.710 

Benefits 

Employer  offers  (%) 

Sick  days  with  full  pay 

25.6 

21.1 

4.5  * 

0.086 

Paid  vacation 

35.5 

31.0 

4.5 

0.119 

Paid  holidays  other  than  Christmas  and  New  Year 

31.7 

28.9 

2.8 

0.324 

A health  plan  or  medical  insurance 

22.6 

19.5 

3.1 

0.221 

None  of  the  above 

24.1 

28.2 

-4.1 

0.131 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

50.9 

54.9 

-4.0 

0.198 

Receive  respect  from  coworkers 

55.4 

55.8 

-0.4 

0.904 

Receive  proper  equipment  needed  to  do  job 

58.7 

61.6 

-2.9 

0.353 

Are  allowed  to  contribute  ideas 

52.9 

55.1 

-2.2 

0.485 

Can  count  on  keeping  job 

26.8 

29.1 

-2.3 

0.417 

Think  job  requires  a lot  of  responsibility 

60.3 

62.0 

-1.7 

0.586 

Think  job  is  physically  demanding 

29.2 

31.6 

-2.4 

0.400 

Risk  health  or  safety 

24.2 

24.6 

-0.4 

0.873 

(continued) 


122 


Table  4.4  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Are  members  of  labor  union  (%) 

18.3 

12.6 

5.6  ** 

0.013 

Have  possibility  of  a promotion  (%) 

37.3 

36.7 

0.6 

0.841 

Duties/reauirements.  at  least  once  Der  month  (%  reporting) 

Reading  and  writing  skills 

48.3 

44.7 

3.6 

0.250 

Work  with  computers 

22.9 

21.6 

1.3 

0.604 

Arithmetic  skills 

31.6 

30.7 

0.9 

0.750 

Customer  contact 

59.2 

60.0 

-0.7 

0.815 

Transportation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

26.3 

21.9 

4.4 

0.105 

0-15 

14.8 

19.4 

-4.5  * 

0.052 

16-30 

19.7 

18.1 

1.6 

0.517 

31-45 

13.5 

11.5 

1.9 

0.355 

46  or  more 

18.6 

18.7 

-0.1 

0.970 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

32.6 

25.8 

6.7  ** 

0.017 

Ever  received  a promotion  (%) 

15.4 

11.4 

4.1  * 

0.057 

Found  a different  job  while  working  (%) 

21.2 

20.4 

0.7 

0.773 

Left  a job  to  go  into  a higher-paying  one  (%) 

20.6 

18.8 

1.7 

0.486 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

21.9 

22.5 

-0.6 

0.827 

Earnings 

24.4 

20.8 

3.6 

0.166 

Benefits 

15.9 

13.5 

2.3 

0.294 

Number  of  hours 

22.1 

17.8 

4.3  * 

0.090 

Start  and  end  of  workday 

22.2 

20.6 

1.6 

0.545 

Commuting  time 

16.9 

20.0 

-3.0 

0.214 

Job  security 

20.4 

18.6 

1.8 

0.484 

Opportunity  to  advance 

17.6 

17.5 

0.1 

0.978 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 
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who  had  ever  received  a promotion.  In  addition,  the  program  increased  the  percentage  who,  in 
their  current  or  most  recent  job,  were  being  offered  sick  days  with  full  pay  or  who  reported  that 
they  had  experienced  an  improvement  in  the  number  of  hours  that  they  were  working.  Finally, 
the  Chicago  ERA  program  increased,  by  5.6  percentage  points,  the  percentage  of  program 
group  members,  relative  to  control  group  members,  who  were  members  of  labor  unions  at  the 
time  of  the  42 -month  survey. 

Nonetheless,  neither  the  12-month  nor  the  42-month  survey  provided  evidence  that  the 
Chicago  ERA  program  increased  advancement  in  tenns  of  individuals’  job  schedules,  the  skills 
they  used  in  their  jobs,  or  their  work  environments.  In  addition,  despite  evidence  of  advance- 
ment for  program  group  members,  only  23  percent  of  employed  program  group  members  were 
making  over  $9  per  hour  at  the  job  they  held  at  the  time  of  the  42 -month  survey.  This  indicates 
that  the  majority  of  employed  program  group  members  generally  remained  in  low-paying  jobs 
even  after  having  been  eligible  for  the  program. 

Public  Assistance  and  Average  Annual  Income 

Since  a major  goal  of  the  ERA  program  in  Chicago  was  to  reduce  the  proportion  of  the 
TANF  caseload  that  was  in  “stop-the-clock”  status,  TANF  receipt  rates  and  payment  amounts 
are  key  outcomes  of  this  test.  The  Chicago  ERA  program  reduced  the  average  annual  amount  of 
TANF  received  over  the  cumulative  three-year  follow-up  period  by  over  $200  (Table  4.3).16 
Program  group  members  received  an  average  of  23  percent  less  in  TANF  cash  assistance  than 
the  control  group  received. 

The  majority  of  sample  members  in  the  program  and  control  groups  received  TANF  for 
at  least  one  month  in  the  first  year  of  follow-up;  however,  receipt  rates  decreased  substantially 
over  time  for  both  groups.  The  program  accelerated  this  trend,  decreasing  the  percentage  of 
people  who  ever  received  TANF  by  almost  15  percentage  points  in  the  second  year  of  follow- 
up and  by  8 percentage  points  in  the  third  year  of  follow-up,  compared  with  the  control  group 
levels  (Appendix  Table  B.2).  Interviews  with  staff  suggest  that  some  sample  members  in  the 
program  group  may  have  left  TANF  in  order  to  avoid  meeting  the  participation  requirements  of 
ERA,  which  would  have  been  added  to  the  requirements  for  TANF  receipt. 

The  program  increased  the  amount  of  food  stamp  assistance  received  over  the  three- 
year  follow-up  period.  Program  group  members  received  an  average  of  4 percent  more  in  food 
stamps  than  the  control  group  members  during  the  follow-up  period.  Program  group  members 


16 While  there  are  four  years  of  follow-up  earnings  data  available  for  the  ERA  test  in  Chicago,  there  are 
only  three  years  of  public  assistance  data  available.  Income  was  only  calculated  for  the  three  years  for  which  all 
types  of  data  were  available. 
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may  have  been  eligible  for  larger  food  stamp  grants  because  of  their  reduced  use  of  TANF. 
Despite  increases  in  earnings  and  food  stamp  assistance  for  the  program  group,  ERA  did  not 
produce  any  changes  in  average  annual  income  over  the  cumulative  three-year  follow-up 
period. 


Economic  Impact  Findings,  by  Study  Cohort 

Because  much  can  change  over  the  course  of  program  operations  — including  changes 
in  program  services,  the  composition  of  the  sample,  and  the  external  environment  — impacts 
were  examined  separately  by  study  cohort  for  a number  of  the  ERA  programs.  This  analysis 
was  considered  especially  interesting  for  the  Chicago  ERA  program  because  of  the  interruption 
in  program  funding  that  occurred  midway  through  program  operations  and  because  of  the 
growth  in  staff  caseloads  over  time. 

The  employment  retention  and  earnings  gains  generated  by  the  Chicago  ERA  program 
were  concentrated  among  those  who  entered  the  study  in  the  first  half  of  the  enrollment  period 
(the  “early  cohort”)  rather  than  people  who  entered  the  study  in  the  second  half  of  the  enroll- 
ment period  (the  “late  cohort”).17  Program  group  members  in  the  early  cohort  experienced 
gains,  relative  to  the  control  group,  in  most  employment  retention  and  earnings  outcomes  that 
were  examined  in  the  cumulative  four-year  follow-up  period  (Appendix  Table  B.5).18  For  the 
early  cohort,  the  program  increased  the  percentage  of  program  group  members  who  had  an 
employment  spell  of  at  least  four  quarters  by  5 percentage  points,  or  by  9 percent  of  the  control 
group  level.  The  program  also  increased  average  annual  earnings  by  over  $900,  or  by  1 5 percent 
of  control  group  earnings.  In  addition,  it  increased  the  percentage  of  quarters  in  which  program 
group  members  were  earning  $3,500  or  more  by  over  4 percentage  points,  or  by  25  percent  of 
the  control  group  level.  The  program  did  not  produce  any  statistically  significant  impacts  on  the 
main  measures  of  employment  retention  or  earnings  for  the  late  cohort. 

The  42-month  survey  also  provides  evidence  that  the  Chicago  ERA  program  was  more 
effective  for  the  early  cohort  than  for  the  late  cohort.  For  several  outcomes  measured  in  this 
survey,  program  impacts  for  the  early  cohort  are  statistically  significant,  even  when  the  impacts 
for  the  full  sample  are  not.  For  the  early  cohort,  the  program  increased  the  percentage  of 


17Because  control  group  earnings  were  lower  for  the  early  cohort,  subgroup  impacts  were  ran  to  see 
whether  the  cohort  effects  were  due  to  the  changing  composition  of  the  sample  over  time.  This  analysis  found 
that  the  program  produced  gains  only  for  people  with  below-average  earnings  in  the  quarter  prior  to  study  entry 
who  were  in  the  early  cohort.  The  program  did  not  produce  gains  for  these  low  earners  in  the  late  cohort.  Thus, 
it  does  not  appear  that  the  cohort  effects  are  the  result  of  the  changing  composition  of  the  sample  over  time. 

There  are  statistically  significant  differences  between  the  economic  impacts  for  the  early  and  late  cohorts 
of  sample  members. 
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program  group  members  who  were  working  between  35  and  44  hours  per  week  at  the  job  they 
held  at  the  time  of  the  42-month  survey  by  9 percentage  points,  or  by  almost  28  percent  of  the 
control  group  level  (Appendix  Table  B.6).  The  Chicago  ERA  program  also  produced  gains  for 
the  program  group,  relative  to  the  control  group,  in  the  percentage  of  early  cohort  sample 
members  who  reported  having  various  job  benefits  in  the  job  they  held  at  the  time  of  the  survey; 
increases  were  found  in  eligibility  for  paid  sick  days,  paid  vacation,  paid  holidays,  and  a health 
plan  or  medical  insurance. 

The  study  cohorts  also  show  different  patterns  of  impacts  on  public  assistance  receipt 
and  income  (Appendix  Table  B.5).  Both  the  early  and  the  late  cohort  experienced  a decline  in 
TANF  assistance,  relative  to  the  control  group,  despite  the  fact  that  there  were  no  gains  in 
employment  for  the  late  cohort.  This  provides  additional  support  to  the  idea  that  part  of  the 
decline  in  TANF  assistance  was  due  to  program  group  members’  leaving  TANF  to  avoid  the 
additional  ERA  participation  requirements.  The  decline  in  TANF  assistance  for  the  late  cohort 
was  largely  offset  by  an  increase  in  food  stamp  assistance,  and  the  program  did  not  have  an 
effect  on  average  annual  income  for  this  cohort.  While  there  were  no  statistically  significant 
increases  in  food  stamp  receipt  for  the  early  cohort,  the  program  did  increase  average  annual 
income  relative  to  the  control  group,  by  an  average  of  $650  per  year  over  the  first  three  years  of 
follow-up. 

Several  factors  may  explain  why  the  Chicago  ERA  program’s  beneficial  effects  on  eco- 
nomic outcomes  were  concentrated  in  the  early  cohort.  This  pattern  of  impacts  could  be  related 
to  the  interruption  in  program  funding  that  occurred  roughly  two  years  into  program  operations 
(discussed  above).  As  a result,  individuals  who  were  randomly  assigned  in  the  last  months  of 
the  sample  intake  period  received  more  limited  exposure  to  the  program  than  those  who  were 
assigned  earlier.19  Another  factor  that  could  have  contributed  to  these  findings  is  that  staff 
caseloads  grew  substantially  over  time,  which  may  have  affected  service  delivery  in  the  later 
phases  of  the  project.  Finally,  the  composition  of  the  ERA  sample  changed  over  time.  Individu- 
als who  were  randomly  assigned  later  in  the  follow-up  period  had  higher  earnings  and  employ- 
ment levels,  which  may  have  created  a higher  “bar”  for  impacts. 

Chicago  ERA  Program:  Summary  and  Conclusions 

When  ERA  began,  Illinois  had  existing  policies  that  supported  low-wage  work:  a rela- 
tively generous  earned  income  disregard  policy  that  encouraged  employment  while  remaining 
eligible  for  TANF,  and  a policy  whereby  months  of  working  and  receiving  TANF  did  not  count 


l9The  12-month  survey  sample  came  entirely  from  the  late  cohort,  so  differences  in  participation  between 
the  study  cohorts,  which  require  survey  data,  cannot  be  examined. 
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toward  individuals’  TANF  time  limit.  As  a result,  a growing  number  of  recipients  remained  in  a 
“stop-the-clock”  status  for  long  periods.  In  the  Chicago  ERA  program,  the  Illinois  Department 
of  Human  Services  sought  to  help  these  employed  recipients  advance  into  higher-paying  jobs, 
both  to  improve  the  clients’  quality  of  life  and  to  reduce  the  state’s  TANF  caseload.  Compared 
with  the  limited  and  voluntary  postemployment  services  previously  available  to  working 
individuals  receiving  TANF,  the  Chicago  ERA  program  was  a mandatory  and  more  compre- 
hensive model,  emphasizing  work-focused  retention  and  advancement.  Operated  by  a for-profit 
organization  with  strong  linkages  to  local  employers,  the  Chicago  ERA  model  incorporated  a 
combination  of  services,  including  ongoing  staff-client  relationships,  financial  incentives  to 
participate,  targeted  job  search  assistance,  and  help  identifying  and  accessing  employers  and 
jobs  with  opportunities  for  advancement.  Although  no  formal  advancement  services  were 
provided  to  the  control  group,  they  could  access  any  services  available  in  the  community. 

Field  research  showed  that  the  Chicago  ERA  program  provided  a clearly  defined  set  of 
advancement-focused  services  and  that  the  most  common  approach  was  to  help  individuals 
move  fairly  quickly  to  a new  job  that  paid  somewhat  more  than  their  current  job.  At  the  same 
time,  the  ERA  program  experienced  some  difficulty  providing  advancement  services,  given 
unexpected  levels  of  job  loss  among  program  group  members.  ERA  staff  were,  however,  well 
equipped  to  turn  job  loss  into  an  opportunity  for  clients  to  move  toward  their  career  goals. 

More  than  half  of  all  program  group  members  had  contact  with  ERA  staff  or  staff  from 
an  employment  program  within  the  year  following  study  entry.  Over  a third  of  program  group 
members  reported  receiving  help  with  retention  and  advancement  over  this  same  period,  with  a 
quarter  receiving  help  finding  a better  job  while  working. 

Compared  with  control  group  members,  a far  larger  proportion  of  the  program  group 
reported  contact,  and  ongoing  contact,  with  ERA  staff  or  staff  from  an  employment  program 
within  the  year  following  study  entry.  Furthermore,  the  Chicago  ERA  program  tripled  the  rate 
at  which  program  group  members  participated  in  retention  and  advancement  services,  compared 
with  control  group  levels  (which  were  quite  low).  The  program  increased  both  the  percentage  of 
program  group  members  who  participated  in  job  search  activities  and  the  percentage  who 
received  help  finding  a better  job  while  working,  by  over  20  percentage  points  relative  to  the 
control  group  levels. 

The  Chicago  ERA  program  produced  gains  in  several  measures  of  employment  reten- 
tion, earnings,  and  advancement  in  the  cumulative  four-year  follow-up  period.  The  program 
appears  to  have  assisted  individuals  to  move  into  Ul-covered  employment  from  non-UI-covered 
employment,  to  become  reemployed  more  quickly  after  loss  of  employment,  and,  possibly,  to 
move  to  better  jobs  when  they  were  already  employed.  The  program  also  generated  positive 
effects  on  many  broad  measures  of  advancement  — such  as  job-offered  benefits  and  union 
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membership.  Earnings  gains  were  larger  for  those  who  entered  the  study  in  the  first  half  of  the 
enrollment  period  (the  early  cohort)  than  the  earnings  gains  for  the  program  group  as  a whole. 
While  the  program  increased  earnings  and  the  amount  of  food  stamp  assistance  received,  it 
decreased  the  amount  of  TANF  received,  compared  with  the  control  group  levels.  Some 
recipients  appear  to  have  left  TANF  to  avoid  the  ERA  program’s  participation  requirement. 

Several  factors  likely  contributed  to  the  economic  effects  of  the  Chicago  ERA  program. 
The  program  was  strongly  implemented,  and  it  increased  participation  in  employment  retention 
and  advancement  services,  relative  to  the  levels  of  participation  for  the  control  group.  The 
Chicago  ERA  service  provider  appeared  well  suited  to  implement  this  particular  type  of 
program  model  because  of  its  strong  relationships  with  many  local  employers.  The  program’s 
requirement  that  individuals  participate  in  postemployment  services  (a  requirement  that  the 
control  group  members  did  not  face)  and  its  incentives  for  participation  may  also  have  contri- 
buted to  its  increases  in  participation  in  services  beyond  the  levels  of  the  control  group. 

The  results  of  this  test  show  that  it  is  possible  to  help  employed  TANF  recipients  — 
particularly  those  working  outside  the  formal  labor  market  — advance  in  the  labor  market.  The 
results  also  suggest  that  requiring  employed  recipients  to  participate  in  retention  and  advance- 
ment services  can  generate  substantial  reductions  in  TANF  receipt. 
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Los  Angeles  Reach  for  Success  (RFS)  ERA  Findings 


Los  Angeles  RFS  ERA  at  a Glance 

• Targeted  recently  employed  (full-time)  TANF  recipients 

• Provided  strongly  marketed,  individualized,  and  flexible  postemployment  services 

• Was  operated  by  the  TANF  agency 

• Experienced  difficulties  with  unexpectedly  high  reemployment  needs  and  limited  staff 
skills  in  providing  career  counseling 

• Was  compared  with  a control  group  that  participated  in  a less  intensive  postemployment 
program 

• Produced  limited  increases  in  the  use  of  retention  and  advancement  services 

• Did  not  increase  employment  retention  or  advancement 


Los  Angeles  RFS  ERA  Test:  Introduction 

The  Los  Angeles  RFS  ERA  program,  which  operated  from  March  2002  through  June 
2005,  aimed  to  promote  job  retention  and  advancement  among  employed  individuals  who  were 
receiving  TANF  by  providing  them  with  individualized  and  flexible  retention  and  advancement 
services.  The  study  enrolled  sample  members  from  July  2002  through  June  2004. 

Origins  of  the  Test 

When  ERA  began,  TANF  clients  in  California  were  required  to  work  full  time  (at  least 
32  hours  per  week)  in  order  to  receive  benefits,  unless  they  were  otherwise  exempt.  California 
also  provided  relatively  high  TANF  grant  levels  and  disregarded  a relatively  large  portion  of 
earnings  when  determining  the  grant  levels  for  those  who  were  working.  As  a result,  many 
clients  who  secured  full-time  work  were  still  eligible  for  a cash  grant. 

In  the  late  1990s,  Los  Angeles  County  Department  of  Public  Social  Services  (DPSS) 
started  to  provide  some  small-scale,  postemployment  services  to  help  full-time  working  TANF 
recipients  sustain  their  employment  and  increase  their  earnings.  These  services  — for  which  the 
control  group  in  the  ERA  test  was  eligible  once  the  study  started  — included  work  supports  for 
full-time  employed  TANF  recipients,  most  commonly,  child  care  and  transportation  assistance. 
In  addition,  individuals  were  encouraged  to  pursue  education  and  training  and  were  provided 
with  tuition  assistance  if  they  were  interested.  (DPSS  staff  did  not  pursue  work-based  strategies 
for  advancement  with  these  individuals  or  provide  counseling  on  retention  issues.) 
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DPSS  assessments  of  participation  levels  in  these  postemployment  services,  however, 
indicated  that  the  take-up  rate  was  low.  In  response,  DPSS  developed  the  Reach  for  Success 
(RFS)  program,  which  sought  to  increase  the  take-up  of  services  by  full-time  employed  TANF 
recipients  through  implementing  an  intensive  and  multifaceted  outreach  and  marketing  strategy 
as  well  as  offering  highly  individualized  services  that  could  be  provided  outside  the  welfare 
office  and  during  evenings  and  weekends.  To  allow  for  these  expanded  activities,  the  caseloads 
of  staff  in  the  RFS  program  were  set  at  roughly  half  the  size  of  the  caseloads  of  staff  in  the 
control  group  program. 

Labor  Market  Context 

At  the  time  that  random  assignment  for  the  Los  Angeles  RFS  test  began,  the  Los  An- 
geles area’s  labor  market  was  still  suffering  the  effects  of  the  2001  recession,  with  a rising 
unemployment  rate  and  declining  job  levels.  As  with  the  national  trend,  the  area’s  labor  market 
began  to  improve  after  2003.  By  2005,  the  unemployment  rate  and  number  of  jobs  had  returned 
to  prerecession  levels,  indicating  a faster  recovery  rate  for  the  area  than  for  the  nation  as  a 
whole.  Thus,  while  the  economic  environment  was  difficult  at  the  beginning  of  the  Los  Angeles 
RFS  test,  conditions  were  slightly  better  than  national  levels  for  much  of  the  program’s  opera- 
tion and  follow-up  period. 

Target  Population 

The  Los  Angeles  RFS  ERA  program  targeted  TANF  recipients  who  had  been  working 
(generally  for  30  days)  in  a full-time  job  for  32  hours  or  more  per  week  and  were  living  in  three 
of  Los  Angeles  County’s  seven  administration  regions.  Table  4.2  shows  selected  characteristics 
of  the  Los  Angeles  RFS  ERA  sample  members  as  of  their  study  entry.  About  half  the  sample 
members  reported  that  they  had  received  TANF  for  more  than  two  years  (cumulatively)  as  of 
study  entry,  and  about  half  had  at  least  a high  school  diploma,  a GED  certificate,  or  a higher 
education  credential. 

Significance  of  This  Test 

The  Los  Angeles  RFS  test  examines  whether  an  intensely  marketed,  individualized,  and 
flexible  advancement  program,  administered  by  a welfare  agency,  can  increase  service  receipt 
by  employed  TANF  recipients  and  move  them  into  higher-paying  jobs,  compared  with  a less 
intensive  and  more  “rule-bound”  postemployment  program,  also  provided  by  welfare  agency 
staff.  Chart  4.2  provides  a succinct  presentation  of  important  differences  between  the  program 
and  control  group  situations. 
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The  Employment  Retention  and  Advancement  Project 
Chart  4.2 

Program-Control  Group  Differences: 

Los  Angeles  RFS  Test 


ERA  Group 

Control  Group 

Program 
resources  and 
structures 

• Operated  by  public  welfare  agency 

• Operated  by  public  welfare  agency 

Staffing 

• Staff  exclusively  responsible  for  ERA  clients 

• Flexible  staff  schedules  and  meeting  locations; 
flexibility  for  clients 

• Staff  responsible  for  both  unemployed  and 
employed  clients 

• Traditional  staff  schedules  and  locations; 
limited  flexibility 

Staff-client 

engagement 

• Proactive  and  dynamic  outreach  and 
marketing  strategies 

• Services  tailored  to  participants 

• Minimal  outreach,  but  achieved  similar 
rates  of  contact  as  ERA 

• Less  individualized  services 

• Limited  use  of  employment  plans 

• Monthly,  ongoing  contacts 

• No  consistent  use  of  employment  plans. 

• Quarterly  ongoing  contacts;  primarily 
compliance-focused 

Retention 

services 

• Reemployment  services  prioritized  better  jobs 
(wages,  benefits,  and  career  prospects)  and 
available  for  30-45  days  after  job  loss 

• Provision  of  work  supports  and  social  service 
referrals  and  as-needed  assistance  accessing 
benefits 

• Reemployment  services  prioritized  finding 
any  job  and  available  for  30  days  after  job 
loss;  achieved  comparable  proportions  of 
people  participating  in  job  search  activities 
as  ERA 

• Provision  of  work  supports  and  social 
service  referrals  with  miminal  assistance 
accessing  benefits 

Advancement 

services 

• Frequent  education  and  training  referrals 

• Limited  education  and  training  referrals, 
triggered  by  client  initiation,  but  achieved 
similar  take-up  rates  as  ERA 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 
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Los  Angeles  RFS  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

The  Los  Angeles  RFS  ERA  program  was  operated  by  Los  Angeles  DPSS  in  three  of 
the  Los  Angeles  DPSS’s  seven  regional  welfare-to-work  offices  and  was  funded  by  TANF 
resources.  Although  DPSS  did  not  work  directly  with  other  organizations  to  operate  the  pro- 
gram, staff  occasionally  referred  participants  to  outside  agencies  for  services,  including  mental 
health  and  substance  abuse  treatment,  domestic  violence  services,  and  housing  assistance. 

Staffing 

Staff  in  “generic”  staff  positions  primarily  delivered  RFS  program  services,  and  the 
program  group  members  were  their  sole  responsibility  (rather  than  also  serving  clients  in  the 
existing  postemployment  program,  for  example).  There  were  five  or  six  of  these  positions  per 
office  location.  The  staff  in  each  regional  office  also  included  a job  developer,  who  was  primari- 
ly responsible  for  working  with  program  group  members  who  lost  their  jobs.  Originally,  the  job 
developers  were  to  focus  on  helping  employed  clients  find  better  jobs;  in  reality,  they  spent  a 
very  small  portion  of  their  time  on  this.  Finally,  ERA  unit  supervisors  in  each  office  directly 
monitored  day-to-day  activities,  focusing  primarily  on  compliance  with  larger  welfare-to-work 
program  rules  about  work  participation  and  reporting  earnings. 

Although  Los  Angeles  RFS  ERA  staff  were  based  in  their  regional  DPSS  offices,  they 
occasionally  met  with  program  group  members  off-site  and  after  hours  in  order  to  facilitate 
engagement  and  ongoing  participation.  Responsibilities  and  caseload  sizes  (from  40  to  85 
clients)  varied  across  staff  and  regions  and  over  time.  There  was  some  degree  of  specialization; 
for  example,  some  select  staff  worked  with  Spanish-speaking  clients  or  with  clients  who  had 
particular  service  needs.  Finally,  although  ERA  staff  did  not  have  specific  performance  stan- 
dards (or  performance  incentives),  participation  and  employment  outcomes  for  program  group 
members  were  monitored.  Program  staff,  for  the  most  part,  did  not  have  previous  experience  or 
training  in  career  counseling.  As  was  the  case  in  the  majority  of  ERA  programs,  MDRC 
provided  training  and  technical  assistance  to  help  ERA  staff  develop  tools  that  they  could  use  to 
help  participants  advance.  This  training  focused  on  engaging  and  motivating  program  group 
members  and  working  with  them  to  outline  very  specific  steps  to  take  toward  advancement. 

Staff-Client  Engagement 

Highly  individualized  and  flexible,  ongoing  staff-client  relationships  were  central  to  the 
Los  Angeles  RFS  ERA  program.  In  order  first  to  engage  individuals,  program  staff  did  intense 
outreach  and  marketing,  employing  various  strategies  that  evolved  over  time.  This  was  partially 
necessitated  by  the  fact  that  ERA-specific  activities  did  not  introduce  any  additional  mandatory 
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TANF  participation  requirements.20  The  full-time  work  schedules  of  program  group  members 
may  have  also  made  such  outreach  efforts  critical. 

Despite  outreach  efforts,  engagement  was  a challenge,  according  to  ERA  staff.  Field  re- 
search suggests  that  engagement  in  the  program  may  have  been  hampered  by  a reluctance  of 
TANF  recipients,  in  some  cases,  to  be  involved  with  a welfare  program  after  they  were  work- 
ing. Table  4.5  shows  that  about  half  (54  percent)  of  program  group  members  reported  that  they 
had  any  contact  with  staff  in  an  employment  program  during  the  first  year  after  they  entered  the 
study;  however,  this  rate  is  no  different  from  the  level  of  contact  with  employment  program 
staff  that  was  reported  by  control  group  members. 

Once  contact  with  program  group  members  was  made,  ERA  staff  developed  employ- 
ment plans  with  participants,  based  on  an  informal  assessment  in  initial  meetings.  With  these 
plans,  ERA  staff  directed  action  steps  and  service  options  (such  as  participating  in  education  or 
training  services  or  providing  work  supports)  for  program  group  members.  At  the  program’s 
outset,  ERA  staff  lacked  experience  in  some  of  these  areas,  such  as  career-oriented  counseling. 
Staff  did  develop  some  skills  over  time  through  experience  and  through  county-  and  MDRC- 
facilitated  professional  training.  The  program  did  result  in  a higher  take-up  of  career  assessment 
services  by  the  program  group  than  the  control  group,  although  the  difference  is  small:  the  12- 
month  survey  indicates  that  almost  1 5 percent  of  program  group  members  reported  receiving 
help  with  career  assessment,  compared  with  10  percent  of  control  group  members  (Table  4.5). 

Maintaining  individuals’  participation  in  the  Los  Angeles  RFS  program  was  difficult.  In 
field  research  interviews,  ERA  staff  described  themselves  as  more  likely  to  work  with  individu- 
als who  were  motivated  to  participate,  and  they  reported  that  they  found  it  difficult  to  engage 
individuals  who  were  less  interested  in  pursuing  advancement.  Among  those  individuals  with 
whom  ERA  or  control  group  staff  were  actively  working,  field  research  indicates  that  the 
intensity  of  the  staff-client  relationship  was  greater  with  program  group  members  than  with 
control  group  members  and  that  contacts  were  more  personalized  and  substantive  with  program 
group  members.  According  to  the  12-month  survey,  only  a fifth  of  program  group  members  had 
had  recent  contact  with  staff  of  an  employment  program.  There  were  no  differences  in  the  rate 
of  recent  contact  or  average  contacts  with  staff  between  program  and  control  group  members 
(Table  4.5). 


20The  work  requirement  imder  TANF  rules  was  32  hours  per  week.  Therefore,  in  the  event  of  job  loss, 
individuals  had  30  days  to  find  another  job.  At  that  point,  they  lost  eligibility  for  postemployment  services  and 
were  required  to  return  to  the  TANF  preemployment  program.  The  ERA  staff,  however,  had  flexibility  in 
enforcing  this  requirement  if  clients  were  making  good  progress  in  their  job  search  as  of  30  days. 
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The  Employment  Retention  and  Advancement  Project 
Table  4.5 

Summary  of  Impacts: 


Los  Angeles  RFS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

54.0 

50.5 

3.5 

0.304 

Average  number  of  contacts  with  staff 

7.3 

6.2 

1.1 

0.252 

Talked  with  staff  in  past  4 weeks  (%) 

20.3 

18.9 

1.4 

0.605 

Retention  services 

Participated  in  a job  search  activity  (%) 

60.4 

55.2 

5.2 

0.130 

Received  help  dealing  with  problems  on  the  job  (%) 

5.5 

3.7 

1.8 

0.221 

Advancement  services 

Received  help  with  career  assessment  (%) 

14.4 

9.9 

4 5** 

0.045 

Received  help  finding  a better  job  while  working  (%) 

7.9 

5.8 

2.2 

0.221 

Participated  in  an  education/training  activity3  (%) 

35.9 

34.2 

1.7 

0.602 

Received  any  help  with  retention/advancement"  (%) 

21.6 

17.5 

4.1 

0.134 

Sample  size  (total  = 848) 

428 

420 

Cumulative  economic  outcomes  f Years  1-3) 

Employment  measures 

Ever  employed  (%) 

92.3 

91.5 

0.8 

0.279 

Average  quarterly  employment  (%) 

63.6 

64.2 

-0.6 

0.453 

Had  employment  spell  of  at  least  4 quarters  (%) 

70.5 

70.6 

-0.2 

0.893 

Length  of  longest  employment  spell,  in  quarters 

6.7 

6.7 

0.0 

0.931 

Length  of  longest  unemployment  spell,  in  quarters 

3.8 

3.7 

0.1 

0.556 

Earning  and  advancement  measures 

Average  annual  earnings  (S) 

9,833 

9,837 

4 

0.985 

Quarters  with  earnings  of  $3,500  or  more  (%) 

32.9 

33.6 

-0.7 

0.427 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  3 (%) 

Not  employed  in  one  or  both  years 

32.6 

30.9 

1.7 

0.159 

Earnings  decreased  by  more  than  $250 

18.9 

19.3 

-0.4 

0.671 

Earnings  changed  by  less  than  $250 

6.5 

7.4 

-0.9 

0.191 

Earnings  increased  by  $250  or  more 

41.8 

42.2 

-0.4 

0.752 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

3,313 

3,216 

97 

0.137 

Average  annual  food  stamps  received  ($) 

2,055 

2,012 

43 

0.244 

Average  annual  income3  ($) 

15,202 

15,065 

137 

0.473 

Sample  size  (total  = 5,700) 

2,857 

2,843 

(continued) 


134 


Table  4.5  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

cThis  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


Initial  Job  Preparation  and  Placement  Services 

Initial  job  preparation  and  placement  services  were  not  a priority  of  the  Los  Angeles 
RFS  ERA  program,  inasmuch  as  its  target  population  was  TANF  recipients  who  were  already 
employed  (and,  therefore,  were  assumed  not  to  need  initial  job  placement).  Job  search  services 
in  the  context  of  reemployment  and  advancement  are  discussed  below. 

Retention  Services 

Job  loss  was  common  in  the  Los  Angeles  RFS  ERA  program  (and  also  among  control 
group  members),  and  a large  portion  of  ERA  resources  were  directed  to  job  search  services  for 
reemployment.  However,  there  were  no  differences  between  program  and  control  group 
members  in  tenns  of  their  participation  in  job  search  activities  during  the  first  year  after  study 
entry  (according  to  the  survey;  Table  4.5).  According  to  field  research,  the  approach  to  reem- 
ployment differed  slightly  for  program  and  control  group  members,  however,  in  that  the 
services  for  ERA  program  group  members  focused  on  securing  them  better  jobs  (in  terms  of 
wages,  benefits,  or  career  prospects)  while  the  services  for  control  group  members  prioritized 
finding  any  job.  Also,  as  suggested  above,  ERA  job  developers  worked  with  unemployed 
program  group  members  a little  longer  than  the  30-day  limit  to  which  staff  working  with  the 
control  group  had  to  adhere. 

Beyond  job  search  services  for  reemployment,  retention  services  in  the  RFS  ERA  pro- 
gram included  connections  to  other  services,  such  as  child  care  and  transportation  (and  as- 
needed  help  securing  such  benefits),  aimed  at  supporting  ongoing  employment.  The  ERA 
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program  also  provided  help  with  ancillary  costs  for  such  things  as  books,  school  fees,  and 
clothing  for  work  or  school.  In  the  12-month  survey,  more  than  half  of  program  group  members 
reported  receiving  help  with  support  services  — a greater  proportion  than  in  the  control  group 
(Appendix  Table  B.8).  Similarly,  almost  two-thirds  of  program  group  members  reported 
receiving  help  accessing  public  benefits  — also  a greater  proportion  than  in  the  control  group. 

Advancement  Services 

The  Los  Angeles  RFS  ERA  program  used  both  work-based  and  education-  or  training- 
focused  strategies  in  its  advancement  approach.  In  seeking  to  help  people  advance,  customer 
service  and  flexibility  were  the  cornerstones  of  the  approach,  and  staff  supported  and  encour- 
aged participants  to  pursue  their  own  career  goals.  Despite  being  a priority  of  the  ERA  program, 
advancement  services  consumed  relatively  little  staff  time,  because  reemployment  needs 
consumed  a higher  level  of  program  staff  resources  than  expected. 

Depending  on  the  individual’s  employment  plans,  ERA  staff  sometimes  encouraged 
and  directed  program  group  members  to  education  and  training  providers.  The  12-month  survey 
indicates  that  a little  more  than  a third  of  program  group  members  reported  that  they  had 
participated  in  education  and  training  activities  — a proportion  no  greater  than  that  of  the 
control  group  (Table  4.5).  The  42 -month  survey,  however,  suggests  that  one  type  of  increase 
eventually  occurred:  program  group  members  were  more  likely  than  control  group  members  to 
report  in  the  later  survey  that  they  had  attended  a college  class  since  study  entry  (not  shown).21 

Employer  Linkages 

While  the  Los  Angeles  RFS  ERA  program  included  a work-focused  approach  to  ad- 
vancement, there  were  no  formal  connections  between  ERA  staff  and  local  employers.  ERA  job 
developers  did  visit  local  employers,  however,  and  they  developed  relationships  with  them  to 
better  understand  their  employees’  job  responsibilities  and  to  identify  job  openings. 

Los  Angeles  RFS  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Los  Angeles  RFS  program  can  be  summarized  as  follows: 

• The  Los  Angeles  RFS  ERA  program  did  not  produce  gains  in  the  main 
measures  of  employment,  employment  retention,  earnings,  or  advancement, 
relative  to  the  levels  seen  in  the  control  group. 


2 'Anderson,  Freedman,  and  Flamilton  (2009). 
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• ERA  also  did  not  have  any  effects,  relative  to  the  control  group  levels,  on  the 
amount  of  TANF  or  food  stamp  assistance  received  or  on  average  annual  in- 
come over  the  three- year  follow-up  period. 

Control  Group  Outcomes 

Average  quarterly  employment  and  average  earnings  for  control  group  members  in  the 
Los  Angeles  RFS  ERA  test  were  comparable  to  the  levels  seen  in  control  groups  in  the  other 
programs  serving  individuals  who  were  already  employed  at  study  entry  (presented  in  Chapters 
4 and  5).  Average  quarterly  employment  was  almost  65  percent,  and  average  earnings  were  just 
under  $10,000  per  year  for  control  group  members  over  the  three-year  follow-up  period  (Table 
4.5).  Average  earnings  increased  relatively  steadily  over  the  follow-up  period  without  a corres- 
ponding increase  in  employment,  which  may  indicate  that  there  was  some  advancement  over 
time  in  the  control  group  (Appendix  Table  B.9).  While  most  of  the  control  group  received  at 
least  one  TANF  payment  in  the  first  follow-up  year,  in  the  third  year  of  follow-up,  54  percent  of 
the  control  group  received  at  least  one  TANF  payment  (Appendix  Table  B.9). 

Employment  and  Employment  Retention 

The  Los  Angeles  RFS  ERA  program  did  not  generate  any  statistically  significant  im- 
pacts, relative  to  the  control  group,  on  any  measure  of  employment  or  employment  retention 
shown  in  Table  4.5  for  the  three-year  follow-up  period.  However,  as  discussed  further  below, 
ERA  did  produce  some  gains  in  measures  of  employment  during  the  first  year  of  follow-up. 

Earnings  and  Advancement 

ERA  did  not  increase  earnings,  and  there  is  no  evidence  that  it  led  to  any  additional  ad- 
vancement, compared  with  the  levels  seen  in  the  control  group.  There  were  some  indications  of 
early  effects  of  the  program  both  in  the  12-month  survey  data  and  in  the  unemployment  insur- 
ance (UI)  data.  According  to  the  12-month  survey,  ERA  produced  some  gains  in  employment  at 
the  time  of  the  12-month  survey  (Appendix  Table  B.l  1).  In  addition,  looking  at  UI  data,  there 
were  statistically  significant  impacts  on  earnings  early  in  the  follow-up  period  for  a cohort  of 
individuals  who  enrolled  early  in  the  Los  Angeles  RFS  ERA  test. 

The  early  impacts  were  not  large  enough  or  persistent  enough  to  produce  improvements 
in  any  of  the  main  measures  of  employment,  employment  retention,  earnings,  or  advancement 
over  the  cumulative  three-year  follow-up  period.  However,  the  early  positive  signs  led  MDRC 
to  field  a 42 -month  survey  to  examine  the  longer-term  effects  of  the  Los  Angeles  RFS  ERA 
program.  As  shown  in  Table  4.6,  this  longer-term  survey  showed  that  the  program  had  not 
produced  gains  in  most  measures  of  employment,  advancement,  or  job  characteristics  at  the 
time  of  the  survey. 
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The  Employment  Retention  and  Advancement  Project 
Table  4.6 

Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42-Month  Survey: 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

91.2 

89.0 

2.2 

0.244 

Currently  employed 

60.0 

59.2 

0.8 

0.791 

No  longer  employed 

31.2 

30.1 

1.1 

0.704 

Hourlv  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

2.3 

0.9 

1.4 

0.114 

$5.00  - $6.99 

3.6 

4.4 

-0.8 

0.567 

$7.00  - $8.99 

16.6 

15.6 

1.1 

0.664 

$9.00  or  more 

36.3 

37.5 

-1.2 

0.708 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

8.0 

4.1 

3 g ** 

0.012 

30-34 

7.2 

7.4 

-0.2 

0.908 

35-44 

36.7 

37.2 

-0.5 

0.874 

45  or  more 

7.9 

10.2 

-2.3 

0.219 

Schedule 

Typical  schedule  (%) 

Regular 

34.6 

38.9 

4.3 

0.164 

Evening  shift 

5.2 

3.6 

1.6 

0.222 

Other  schedule 

18.8 

15.4 

3.4 

0.168 

Workweek  includes  at  least  1 weekend  day  (%) 

28.4 

26.5 

1.8 

0.527 

Benefits 

Employer  offers  (%) 

Sick  days  with  full  pay 

29.9 

30.7 

-0.8 

0.783 

Paid  vacation 

37.6 

35.7 

1.9 

0.538 

Paid  holidays  other  than  Christmas  and  New  Year 

35.4 

32.8 

2.6 

0.391 

A health  plan  or  medical  insurance 

37.2 

35.9 

1.2 

0.690 

None  of  the  above 

14.5 

14.1 

0.3 

0.881 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

48.8 

46.6 

2.2 

0.497 

Receive  respect  from  coworkers 

52.9 

50.4 

2.5 

0.431 

Receive  proper  equipment  needed  to  do  job 

56.0 

53.7 

2.3 

0.462 

Are  allowed  to  contribute  ideas 

52.7 

49.5 

3.2 

0.317 

Can  count  on  keeping  job 

24.4 

25.9 

-1.5 

0.593 

Think  job  requires  a lot  of  responsibility 

55.2 

54.3 

0.9 

0.773 

Think  job  is  physically  demanding 

24.5 

24.3 

0.3 

0.926 

Risk  health  or  safety 

20.6 

19.6 

1.0 

0.703 

(continued) 
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Table  4.6  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Are  members  of  labor  union  (%) 

9.9 

9.5 

0.4 

0.838 

Have  possibility  of  a promotion  (%) 

39.4 

36.3 

3.1 

0.323 

Duties/reauirements.  at  least  once  Der  month  (%  renortinsl 

Reading  and  writing  skills 

44.4 

44.9 

-0.6 

0.861 

Work  with  computers 

30.1 

29.4 

0.7 

0.803 

Arithmetic  skills 

28.0 

28.6 

-0.6 

0.832 

Customer  contact 

49.0 

50.6 

-1.7 

0.604 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

31.2 

30.1 

1.1 

0.704 

0-15 

24.3 

21.1 

3.3 

0.233 

16-30 

22.9 

22.7 

0.2 

0.949 

31-45 

7.1 

7.8 

-0.6 

0.708 

46  or  more 

5.5 

7.8 

-2.3 

0.160 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

37.4 

38.5 

-1.1 

0.715 

Ever  received  a promotion  (%) 

23.2 

22.3 

0.9 

0.728 

Found  a different  job  while  working  (%) 

29.3 

25.8 

3.5 

0.219 

Left  a job  to  go  into  a higher-paying  one  (%) 

24.5 

29.4 

4.9  * 

0.087 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

32.2 

32.8 

-0.5 

0.860 

Earnings 

30.9 

31.6 

-0.8 

0.798 

Benefits 

25.0 

23.8 

1.2 

0.654 

Number  of  hours 

26.2 

29.2 

-3.0 

0.300 

Start  and  end  of  workday 

28.7 

29.1 

-0.4 

0.883 

Commuting  time 

25.4 

23.2 

2.1 

0.452 

Job  security 

27.5 

27.2 

0.2 

0.934 

Opportunity  to  advance 

27.6 

26.7 

0.9 

0.768 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 
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Public  Assistance  and  Average  Annual  Income 

The  Los  Angeles  RFS  ERA  program  had  no  effect  on  the  amount  of  TANF  or  food 
stamp  assistance  received  or  on  average  annual  income  in  the  three  years  of  follow-up,  com- 
pared with  the  control  group  (Table  4.5).  While  the  program  did  increase  the  percentage  of 
program  group  members  receiving  TANF  in  the  second  and  third  years  of  follow-up  (Appendix 
Table  B.9),  the  effects  were  not  large  enough  to  generate  an  increase  in  the  amount  of  TANF 
assistance  received  over  the  cumulative  follow-up  period. 

Los  Angeles  RFS  ERA  Program:  Summary  and  Conclusions 

In  the  late  1990s,  Los  Angeles  County  Department  of  Public  Social  Services  (DPSS)  in- 
itiated a number  of  small-scale,  postemployment  services  to  help  working  TANF  recipients 
sustain  employment  and  increase  their  earnings.  DPSS  assessments  indicated  that  the  take-up 
rate  of  such  activities  was  low.  Designed  to  increase  the  engagement  of  individuals  in  services, 
the  Los  Angeles  RFS  ERA  program  — operated  by  the  county  welfare  agency  — included 
intensive  and  multifaceted  outreach  and  marketing  as  well  as  highly  individualized  staff-client 
relationships  provided  outside  the  welfare  office  setting  and  outside  “normal”  welfare  office 
hours.  TANF  recipients  who  were  randomly  assigned  to  the  control  group  were  eligible  to 
request  the  county’s  standard  postemployment  services  at  their  own  initiative. 

Field  research  suggests  that  customer  service  and  individualized  attention  were  impor- 
tant aspects  of  RFS  services  for  those  individuals  who  engaged  with  the  program.  The  services 
that  were  provided  included  helping  participants  access  work  supports,  assisting  them  with 
reemployment  after  job  loss,  and  developing  work-based  and  education-  or  training-based 
strategies  for  advancement.  Reemployment  activities,  however,  were  a bigger  part  of  the  Los 
Angeles  RFS  ERA  program  than  originally  anticipated,  due  to  clients’  quick  and  high  levels  of 
job  loss,  and  they  often  required  the  redirection  of  staff  time  from  providing  advancement 
services  to  providing  reemployment  services.  In  addition,  staff  often  lacked  training  and 
experience  in  providing  in-depth  career  counseling. 

The  Los  Angeles  RFS  ERA  staff  initially  engaged  many  full-time  working  welfare  re- 
cipients in  the  program,  but  they  had  difficulty  keeping  them  engaged  on  an  ongoing  basis. 
Over  a third  of  program  group  members  reported  participating  in  an  education  or  training 
activity,  but  only  a fifth  of  program  group  members  reported  receiving  any  retention  and 
advancement  services. 

The  RFS  program  did  not  increase  levels  of  participation  in  most  services  among  the 
program  group,  compared  with  the  control  group.  Considering  the  activities  in  which  program 
group  members  were  participating  at  high  rates,  such  as  education  and  training,  control  group 
levels  tended  to  be  quite  high  as  well,  and  the  program  did  not  increase  participation  in  those 
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services.  Considering  other  types  of  services  that  were  generally  not  accessed  by  control  group 
members,  such  as  help  finding  a better  job  while  working,  the  program  still  was  not  able  to 
boost  levels  of  participation,  as  program  group  levels  of  receipt  also  tended  to  be  low.  Relative 
to  the  control  group,  the  program  increased  participation  in  only  one  type  of  retention  and 
advancement  service  — career  assessment — and  this  increase  was  under  5 percentage  points. 

ERA  did  not  generate  any  improvements  in  employment,  employment  retention,  earn- 
ings, or  advancement,  relative  to  the  control  group,  in  the  cumulative  three-year  follow-up 
period.  In  the  end,  despite  the  fact  that  the  Los  Angeles  RFS  ERA  program  was  implemented 
largely  as  designed,  it  did  not  prove  to  be  more  effective  than  control  group  services. 
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Riverside  Work  Plus  and  Training  Focused  (Phase  2)  ERA 
Findings 


Riverside  Phase  2 ERA  at  a Glance 

• Targeted  newly  employed  TANF  recipients 

• Tested  two  education-  and  training-focused  advancement  programs,  one  operated  by  the 
welfare  agency  and  the  other  operated  by  the  workforce  development  agency 

• Provided  referrals  to  education  and  training  programs;  one  program  more  flexible  than 
the  other  in  meeting  TANF  participation  requirements  through  these  activities 

• Compared  both  programs  with  a control  group  that  participated  in  a less  flexible 
program  without  a focus  on  education  and  training 

• Produced  no  substantial  increases  in  education  and  training  participation  for  the  whole 
sample  above  unexpectedly  high  levels  for  the  control  group 

• Resulted  in  no  increases  in  employment  retention  or  advancement 


Riverside  Phase  2 ERA  Test:  Introduction 

The  Riverside  Phase  2 ERA  programs  aimed  to  promote  advancement  among  em- 
ployed TANF  recipients,  using  two  approaches.  The  two  approaches  offered  a different  mix  of 
services,  had  different  participation  requirements,  and  conveyed  different  messages.  However, 
they  shared  the  same  operating  principle:  that  to  advance  in  the  labor  market,  low-wage  workers 
need  to  attain  skills  and  credentials  beyond  what  they  can  acquire  on  the  job.  The  Work  Plus 
program  embodied  one  approach,  and  the  Training  Focused  program  embodied  the  other 
approach;  together,  these  two  programs  are  referred  to  here  as  the  Riverside  “Phase  2”  pro- 
grams, as  they  both  represented  the  second  phase  of  Riverside  County’s  overall  welfare-to-work 
program.  Services  for  the  Riverside  Training  Focused  program  started  in  September  2000  and 
continued  through  October  2006;  the  Riverside  Work  Plus  program  started  in  January  1998  and 
is  still  operating.  Sample  members  enrolled  in  the  study  from  January  2001  to  October  2003. 

Origins  of  the  Test 

In  California,  TANF  clients  are  required  to  work  full  time  (32  hours  per  week)  in  order 
to  receive  benefits  unless  they  are  otherwise  exempt.  California  also  provides  relatively  high 
TANF  grant  levels  and  disregards  a relatively  large  portion  of  earnings  when  determining  the 
grant  levels  for  those  who  are  working.  As  a result,  many  clients  who  secure  full-time  work  are 
still  eligible  for  a cash  grant. 
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The  Riverside  Phase  2 ERA  programs  were  created  in  response  to  the  fact  that,  state- 
wide, many  working  welfare  recipients  who  move  into  employment  often  subsequently  leave 
their  jobs  and  rarely  advance  to  better  jobs.  These  programs  sought  to  improve  access  to 
education  and  training  as  a way  to  promote  the  advancement  of  employed  TANF  recipients. 
The  Riverside  Work  Plus  ERA  program  encouraged  enrollees  to  meet  the  welfare  system’s  quid 
pro  quo  participation  requirements  by  combining  at  least  20  hours  of  employment  per  week 
with  up  to  12  additional  hours  of  attendance  in  remedial  education,  postsecondary  education,  or 
vocational  training.  The  Riverside  Training  Focused  ERA  program  allowed  enrollees  to 
substitute  additional  hours  in  school  or  training  for  hours  on  the  job,  or  even  to  forgo  employ- 
ment temporarily  and  participate  full  time  in  approved  skill-building  activities.  Further,  while 
the  Work  Plus  ERA  program  was  operated  by  the  county  welfare  agency,  the  Training  Focused 
ERA  program  was  operated  by  the  county  workforce  development  agency  (which  also  ran  the 
county’s  Workforce  Investment  Act  [WIA]  One-Stop  Center). 

A program  providing  more  limited  services,  called  “Work  Focused,”  was  available  to 
members  of  the  control  group.  Similar  to  postemployment  programs  run  by  states  and  localities 
(including  Riverside  County)  in  the  mid-  to  late  1990s,  the  Work  Focused  program  made 
available,  on  request,  staff  assistance  related  to  job  retention  and  payments  to  defray  enrollees’ 
child  care,  transportation,  and  other  work-related  expenses.  The  Work  Focused  program  did  not 
encourage  individuals  to  enroll  in  education  or  training  activities. 

Labor  Market  Context 

The  Riverside  Phase  2 ERA  test  began  operating  shortly  before  the  beginning  of  the  na- 
tional 2001  economic  recession.  Prior  to  the  recession,  the  Riverside  area’s  unemployment  rate 
was  considerably  higher  than  the  national  rate.  While  unemployment  did  increase  in  the  area 
after  the  end  of  the  200 1 recession,  the  rate  of  increase  was  much  lower  in  the  area  than  it  was 
nationally.  In  addition,  the  number  of  jobs  in  the  Riverside  labor  market  increased  every  year 
during  the  recession.  Overall,  while  Riverside’s  labor  market  did  show  a downturn  due  to  the 
recession,  for  most  of  the  Riverside  Phase  2 programs’  operation  and  follow-up  period,  condi- 
tions were  positive,  although  competitive,  for  workers. 

Target  Population 

The  Riverside  Phase  2 ERA  programs  targeted  employed  TANF  recipients.  Specifical- 
ly, this  included  individuals  who  were  working  at  least  20  hours  per  week  and  had  held  such 
jobs  for  at  least  30  days  but  who  were  still  receiving  a TANF  grant.  Table  4.2  shows  selected 
characteristics  of  the  Riverside  Phase  2 ERA  sample  members  when  they  entered  the  study. 
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Significance  of  This  Test 

The  Riverside  Phase  2 ERA  test  examines  whether  either  or  both  of  two  education-  and 
training-focused  advancement  programs  (one  with  more  flexibility  for  balancing  work  and 
training  and  based  at  a workforce  development  agency)  can  help  employed  TANF  recipients 
move  into  higher-paying  jobs,  compared  with  a welfare  agency-based  program  that  did  not 
encourage  individuals  to  participate  in  education  and  training  and  did  not  allow  trade-offs 
between  training  and  work  hours  in  meeting  the  state’s  TANF  participation  requirement.  Chart 
4.3  presents  important  differences  between  the  program  and  control  group  situations  in  the 
Riverside  Phase  2 ERA  test. 

Riverside  Phase  2 ERA  Programs:  Features  and  Client  Experiences 

The  two  Riverside  Phase  2 programs  had  the  same  goals  and  target  populations.  This 
section  presents  the  program  features  for  the  Riverside  Phase  2 ERA  programs  overall,  identify- 
ing differences  between  the  Work  Plus  and  Training  Focused  ERA  programs,  as  relevant. 

Program  Resources  and  Organization 

The  Riverside  County  Department  of  Public  Social  Services  (DPSS)  developed  the 
Riverside  Phase  2 ERA  programs.  State  TANF  funds  were  allocated  for  their  administration. 
However,  DPSS  did  not  pay  for  education  and  training  services.  Therefore,  in  the  Work  Plus 
ERA  program,  participants  were  steered  toward  providers  that  possessed  other  funding 
sources  to  pay  for  education  and  training  slots.  In  the  Training  Focused  ERA  program,  other 
funding  sources  (including  a Welfare-to-Work  grant  from  the  U.S.  Department  of  Labor) 
were  used  to  pay  for  some  education  and  training  slots. 

DPSS  operated  the  Work  Plus  ERA  program  from  its  Adult  Services  Division,  which 
also  houses  the  state’s  preemployment  TANF  program,  in  its  1 1 regional  offices.  The  Training 
Focused  ERA  program  was  operated  by  the  Welfare-to-Work  Division  within  the  county’s 
Economic  Development  Agency  (EDA),  which  was  also  the  county’s  Workforce  Investment 
Agency,  in  five  offices. 

Staffing 

The  Work  Plus  ERA  program  used  a multipronged  staffing  structure.  First,  “generic” 
frontline  staff  were  responsible  for  initial  outreach  to  program  group  members  and  for  providing 
various  retention  and  advancement  services  to  them;  they  did  not  provide  preemployment 
services  to  any  clients.  In  contrast,  DPSS  staff  members  serving  control  group  members 
provided  these  individuals  with  postemployment  services  but  also  were  responsible  for  provid- 
ing preemployment  services  to  other  TANF  recipients  who  were  not  involved  in  the  ERA  test. 
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The  Employment  Retention  and  Advancement  Project 
Chart  4.3 

Program-Control  Group  Differences: 
Riverside  Phase  2 ERA  Test 


ERA  Group 

Control  Group 

Program 

• Training  Focused:  Operated  by  public 

resources  and 
structures 

workforce  agency 

• Work  Plus:  Operated  by  public  welfare 
agency 

• Operated  by  public  welfare  agency 

Staffing 

Both  Training  Focused  and  Work  Plus: 

• Staff  primarily  responsible  for  providing  post- 

•  Staff  responsible  for  pre-  and  post- 

employment services 

employment  services  to  TANF  recipients 

Staff-client 

Both  Training  Focused  and  Work  Plus: 

engagement 

• No  service  participation  requirements  beyond 
standard  TANF  work  requirements;  however, 
work-training  flexibility  in  meeting  TANF  work 
requirements 

• Standard  TANF  work  requirements 

• Active  and  multifaceted  outreach 

• No  active  recruitment  of  TANF  recipients 
for  program  services 

• Services  tailored  to  participants 

• Services  not  tailored 

• Development  and  maintenance  of  employment 

• No  employment  plans;  however, 

plan 

comparable  take-up  of  assessment  services 
as  ERA 

• Regular  follow-up  for  compliance  and  further 
service  referrals 

• Periodic,  compliance-oriented  follow-up 

Advancement 

Both  Training  Focused  and  Work  Plus: 

services 

• Flexibility  for  education  and  training  while 

• Education  and  training  not  a priority; 

working 

however,  supports  for  client-initiated 
training 

• Education  and  training  referrals 

• Comparable  take-up  of  education  and 
training  relative  to  Work  Plus  group;  slightly 
less  take-up  relative  to  Training  Focused 
group 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 
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While  these  approximately  65  full-time  Work  Plus  ERA  staff  members  were  primarily 
generalists,  they  informally  developed  specialized  functions.  For  example,  the  Work  Plus  ERA 
program  in  each  participating  DPSS  office  had  a lead  ERA  staff  person  who  carried  relatively 
few  cases  and  had  other  responsibilities,  including  serving  as  a liaison  to  community  colleges. 
Some  Work  Plus  offices  also  had  a technician  who  ran  job  clubs  (job  search  workshops  for 
approximately  10  to  30  clients).  In  addition,  Work  Plus  staff  in  each  office  had  access  to  a 
colocated  DPSS  job  developer,  who  worked  with  both  preemployment  TANF  recipients  and 
ERA  participants. 

Staff  in  the  Training  Focused  ERA  program  also  played  a generalist  role  and  provided  a 
range  of  ERA  services.  Operating  at  a relatively  small  scale,  the  program  had  only  five  staff 
members.  They  could,  however,  call  on  specialized  staff,  such  as  job  developers  from  the 
colocated  WIA  One-Stop  office,  if  they  needed  their  expertise. 

Through  this  staffing  structure  and  related  operating  procedures,  the  Riverside  Phase  2 
ERA  programs  provided  flexibility  for  program  group  members  in  terms  of  meeting  locations 
and  scheduling,  and  staff  held  meetings  in  person  or  over  the  phone. 

Staff-Client  Engagement 

Ongoing  staff-client  relationships  were  at  the  center  of  the  Riverside  Phase  2 ERA  pro- 
grams. Participation  in  the  services  offered  by  the  ERA  programs  was  voluntary,  provided  that 
individuals  met  general  TANF  participation  requirements.  Enrollees  in  both  Riverside  Phase  2 
ERA  programs  (as  was  also  the  case  for  control  group  members)  were  subject  to  California’s 
statewide  TANF  rules,  which  required  recipients  to  work  or  engage  in  approved  employment 
preparation  activities  for  a total  of  32  hours  per  week.  Work  Plus  ERA  enrollees  could  meet  this 
requirement  with  a combination  of  work  and  attendance  in  approved  education  or  training 
activities,  but  they  had  to  work  at  least  20  hours  per  week.  Training  Focused  enrollees  were  able 
to  substitute  additional  hours  in  school  or  training  for  hours  on  the  job  or,  with  approval,  could 
forgo  employment  temporarily  to  participate  full  time  in  education  or  training  activities. 

In  order  to  contact  and  engage  individuals,  the  Riverside  Phase  2 ERA  programs  used 
intensive  outreach  strategies.  These  included  personalized  recruitment  efforts,  which  were 
customized  to  participants’  career  and  family  characteristics.  As  shown  in  Table  4.7,  outreach 
efforts  appear  to  have  paid  off:  according  to  the  12-month  survey,  a greater  proportion  of 
program  group  members  than  control  group  members  (58  percent  versus  50  percent,  respective- 
ly) reported  having  had  contact  with  staff  from  an  employment  program  during  their  first  year 
after  study  entry.  But  these  figures  also  indicate  that  a considerable  proportion  of  program  group 
members  did  not  respond  to  outreach  efforts. 
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The  Employment  Retention  and  Advancement  Project 
Table  4.7 

Summary  of  Impacts: 

Riverside  Phase  2 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact 

P -Value 

Impact  P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

58.2 

58.6 

50.2 

7.9  * 

0.081 

8.3  * 

0.069 

Average  number  of  contacts  with  staff 

9.8 

8.5 

8.0 

1.8 

0.271 

0.5 

0.762 

Talked  with  staff  in  past  4 weeks  (%) 

23.7 

23.6 

21.8 

1.9 

0.617 

1.8 

0.638 

Retention  services 

Participated  in  a job  search  activity  (%) 

62.1 

64.4 

60.3 

1.9 

0.675 

4.1 

0.356 

Received  help  dealing  with  problems  on  the  job  (%) 

4.7 

6.7 

7.1 

-2.4 

0.279 

-0.5 

0.831 

Advancement  services 

Received  help  with  career  assessment  (%) 

13.5 

15.3 

10.5 

3.0 

0.330 

4.8 

0.124 

Received  help  finding  a better  job  while  working  (%) 

9.9 

11.4 

9.5 

0.4 

0.891 

2.0 

0.488 

Participated  in  an  education/training  activity3  (%) 

41.4 

37.3 

31.9 

9.5  ** 

0.033 

5.4 

0.225 

Received  any  help  with  retention/advancement"  (%) 

24.2 

28.9 

21.5 

2.6 

0.513 

7.4  * 

0.066 

Sample  size  (total  = 712) 

234 

237 

241 

Cumulative  economic  outcomes  (Years  1-41 

Employment  measures 

Ever  employed  (%) 

93.9 

94.3 

95.0 

-1.0 

0.372 

-0.6 

0.526 

Average  quarterly  employment  (%) 

60.6 

60.1 

62.1 

-1.4 

0.378 

-2.0 

0.157 

Had  employment  spell  of  at  least  4 quarters  (%) 

73.9 

72.7 

75.3 

-1.5 

0.507 

-2.6 

0.174 

Length  of  longest  employment  spell,  in  quarters 

8.0 

7.9 

8.2 

-0.3 

0.311 

-0.4 

0.113 

Length  of  longest  unemployment  spell,  in  quarters 

5.2 

5.3 

5.1 

0.1 

0.720 

0.2 

0.349 

(continued) 


Table  4.7  (continued) 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Training  Focused 
Group 

Impact  P- Value 

Work  Plus 
Group 

Impact  P-Value 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

9,038 

8,494 

8,873 

166 

0.672 

-379 

0.262 

Quarters  with  earnings  of  $3,500  or  more  (%) 

30.2 

27.4 

28.4 

1.8 

0.240 

-1.0 

0.468 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

34.8 

37.1 

35.5 

-0.8 

0.747 

1.5 

0.462 

Earnings  decreased  by  more  than  $250 

16.3 

18.0 

18.4 

-2.1 

0.286 

-0.4 

0.814 

Earnings  changed  by  less  than  $250 

6.6 

4.1 

5.4 

1.2 

0.299 

-1.4 

0.166 

Earnings  increased  by  $250  or  more 

42.2 

40.8 

40.4 

1.8 

0.464 

0.4 

0.844 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

2,195 

2,060 

2,020 

175  * 

0.092 

40 

0.652 

Average  annual  food  stamps  received  ($) 

1,235 

1,233 

1,205 

31 

0.609 

29 

0.581 

Average  annual  income^  ($) 

12,469 

11,788 

12,098 

371 

0.318 

-310 

0.333 

Sample  size  (total  = 3,029) 

744 

1,532 

753 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative  records. 


NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF  grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language  (ESL)  instruction,  adult  basic  education 
(ABE)  or  General  Educational  Development  (GED)  classes,  college  courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better  job  while  working,  enrolling  in  life 
skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job,  and  addressing  personal  problems. 
cThis  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


After  initially  engaging  program  group  members,  Riverside  Phase  2 ERA  staff  assessed 
participants  to  understand  their  vocational  aptitude  and  interests  and  to  develop  client-specific 
employment  plans,  which  included  identifying  education  and  training  needs  (discussed  further 
below).  Both  Work  Plus  and  Training  Focused  ERA  staff  sought  to  contact  program  group 
members  at  least  once  a month  to  verify  employment  hours  and  arrange  support  services  as 
needed.  Staff  in  both  programs  routinely  monitored  clients’  progress  in  their  activities. 

Initial  Job  Preparation  and  Placement  Services;  Retention  Services 

Neither  job  preparation  and  placement  services  nor  retention  services  were  priorities  of 
the  Riverside  Phase  2 ERA  programs.  Since  the  target  population  was  TANF  recipients  who 
had  held  jobs  for  at  least  30  days,  sample  members  did  not  need  initial  job  placements. 

Employment  instability,  however,  was  a common  problem  in  the  target  population. 
Both  Work  Plus  and  Training  Focused  ERA  staff  had  to  spend  more  time  on  employment 
retention  activities  than  they  had  anticipated.  In  order  to  maintain  ERA  eligibility,  sample 
members  who  became  unemployed  had  to  find  a new  job  within  30  days,  and  failure  to  do  so 
would  result  in  their  transfer  back  to  the  preemployment  TANF  program.22  Newly  unemployed 
program  group  members  typically  received  help  from  ERA  staff  in  the  form  of  job  leads, 
resume  assistance,  and  direction  to  DPSS  resources,  such  as  job  clubs.  According  to  the  12- 
month  survey,  nearly  two-thirds  of  individuals  in  both  Riverside  Phase  2 ERA  programs 
participated  in  a job  search  activity  in  the  first  year  after  study  entry  — a proportion  similar  to 
that  of  the  control  group  (Table  4.7). 

ERA  staff  also  connected  program  group  members  to  other  services,  such  as  child  care, 
transportation  subsidies,  and  ancillary  payments  to  support  participation  in  program  activities. 
They  also  occasionally  made  social  service  referrals  (for  example,  for  mental  health,  substance 
abuse,  and  domestic  violence  services).  According  to  the  12-month  survey,  the  Work  Plus 
program  increased  by  7 percentage  points  the  percentage  of  individuals  who  received  any  type 
of  retention  or  advancement  services,  relative  to  the  control  group;  no  increase  was  found  for 
the  Training  Focused  program  (Table  4.7). 


22This  reemployment  period  was  extended  to  60  days  in  the  fall  of 2001.  Field  research  suggests  that  ERA 
staff  lacked  sufficient  time  to  help  with  reemployment  within  30  days,  as  they  often  discovered  job  loss  several 
weeks  after  it  had  occurred. 
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Advancement  Services 


Advancement  services  were  the  key  feature  of  both  Riverside  Phase  2 ERA  programs. 
The  programs  emphasized  education  and  training  services  to  build  job  skills,  rather  than  job- 
changing strategies. 

Riverside  Phase  2 ERA  staff  regularly  encouraged  participation  in  education  and  train- 
ing programs  as  part  of  their  career  counseling  with  ongoing  clients,  but  their  strategies  for 
doing  so  differed  somewhat.  Program  group  members  in  the  Work  Plus  ERA  program  were 
urged  to  participate  in  education  and  training  on  top  of  their  20  hours  of  required  employment 
per  week,  while  Training  Focused  ERA  program  group  members  were  allowed  to  reduce  their 
required  weekly  employment  hours  to  as  low  as  zero,  replacing  hours  of  work  with  education 
and  training.  In  addition,  Work  Plus  staff  encouraged  program  group  members  to  explore  and 
select  the  education  or  training  institutions  and  programs  that  they  thought  were  most  appropri- 
ate for  themselves,  while  Training  Focused  staff  directed  individuals  to  specific  programs  and 
providers.  Work  Plus  ERA  program  staff  also  identified  providers  that  did  not  require  payment 
for  education  and  training  slots,  while  the  Training  Focused  ERA  staff  could  access  Department 
of  Labor  funding  to  actually  pay  for  slots  if  needed.  Moreover,  Work  Plus  ERA  staff  empha- 
sized developing  individuals’  basic  skills  before  vocational  training,  while  the  Training  Focused 
ERA  staff  supported  addressing  individuals’  basic  education  needs  concurrently  with  skills 
training.  Training  Focused  ERA  staff  also  encouraged  individuals  to  attend  training  courses  for 
up  to  two  years  — longer  than  what  was  encouraged  in  the  Work  Plus  program.  Finally,  Work 
Plus  staff  developed  relationships  with  education  and  training  providers  in  their  communities, 
with  the  intention  of  increasing  the  marketing  of  programs  and  services  to  participants’  particu- 
lar needs,  while  Training  Focused  staff  were  not  as  outreach-oriented  in  this  way. 

Due  to  fluctuations  in  funding  resources  for  education  and  training,  both  the  Work  Plus 
and  the  Training  Focused  ERA  administrators  and  staff  had  some  difficulty  projecting  the 
number  and  type  of  education  and  training  slots  to  which  they  could  refer  participants.  For  part 
of  the  follow-up  period,  a shortage  of  funds  that  was  intended  to  pay  for  specialized  opportuni- 
ties reduced  the  number  of  openings  in  longer-term  training  programs,  especially  for  Training 
Focused  ERA  program  group  members.  Inasmuch  as  alternative  referrals  were  available,  the 
program,  on  the  whole,  was  not  disrupted. 

A fairly  high  proportion  of  control  group  members  attended  education  or  training  activi- 
ties on  their  own  initiative  during  the  year  following  study  entry  (Table  4.7).  This  was  likely 
influenced,  in  part,  by  the  setting  of  Riverside,  where  a large  number  of  public  and  private 
education  institutions  enroll  unemployed  and  low-wage  workers  and  offer  support  for  attendees. 

Considering  all  sample  members,  the  Training  Focused  ERA  program  increased  educa- 
tion and  training  participation  beyond  the  levels  of  the  control  group,  but  the  Work  Plus  ERA 
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program  did  not.  For  the  Training  Focused  program,  the  overall  increase  was  approximately  10 
percentage  points,  and  most  of  the  increase  was  in  participation  in  adult  basic  education  classes 
rather  than  in  training  or  college  classes. 

Slightly  different  results  were  found  by  examining  education  and  training  participation 
increases  for  two  key  subgroups:  those  who,  as  of  study  entry,  either  did  have  or  did  not  have  a 
high  school  diploma  or  a GED  certificate.  Among  sample  members  who  had  a diploma  or  GED 
certificate  at  the  time  of  random  assignment,  the  Work  Plus  and  Training  Focused  programs  had 
little  or  no  effect  on  participation  in  any  type  of  education  or  training.  Among  sample  members 
who  did  not  have  a diploma  or  GED  certificate  at  the  time  of  random  assignment,  both  pro- 
grams increased  participation  in  adult  basic  education  classes:  about  38  percent  of  Training 
Focused  program  group  members  reported  participating  in  basic  education  or  GED  classes  — a 
statistically  significant  increase  of  18  percentage  points  above  the  control  group  average 
(Appendix  Table  B.16).  Similarly,  about  37  percent  of  nongraduates  in  the  Work  Plus  ERA 
program  also  reported  participating  in  basic  education  or  GED  classes  — a 17  percentage  point 
increase  over  the  control  group  level.  For  these  nongraduates,  neither  program  increased 
participation  in  training  or  college  classes. 

In  all  three  program  groups,  only  about  10  percent  to  13  percent  of  all  sample  members 
were  still  participating  in  an  education  or  training  activity  at  the  end  of  the  first  year  of  follow- 
up, according  to  the  12-month  survey,  and  similar  percentages  across  the  three  program  groups 
had  attained  a degree  or  vocational  certificate  by  that  time  (results  not  shown). 

Employer  Linkages 

The  Riverside  Phase  2 ERA  programs  did  not  emphasize  work-based  strategies  for  ad- 
vancement and  did  not  seek  formal  connections  between  staff  and  local  employers.  These 
features  were  also  absent  from  the  DPSS  services  that  were  provided  to  control  group  members. 

Riverside  Phase  2 ERA  Programs:  Economic  Impacts 

Economic  impacts  of  the  Riverside  Phase  2 programs  can  be  summarized  as  follows: 

• Neither  the  Training  Focused  ERA  program  nor  the  Work  Plus  ERA  pro- 
gram generated  any  increases  in  the  main  measures  of  employment,  em- 
ployment retention,  earnings,  or  advancement  in  the  four- year  follow-up  pe- 
riod, relative  to  the  levels  seen  in  the  control  group. 

• The  Training  Focused  ERA  program  increased  the  amount  of  TANF  re- 
ceived but  did  not  have  any  effect  on  the  amount  of  food  stamp  assistance  re- 
ceived or  on  average  annual  income  during  this  time. 
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• The  Work  Plus  ERA  program  did  not  produce  any  changes  in  the  amount  of 
TANF  or  food  stamp  assistance  received  or  in  average  annual  income. 

Control  Group  Outcomes 

Average  quarterly  employment  and  average  earnings  for  control  group  members  in  the 
Riverside  Phase  2 ERA  study  are  comparable  to  the  levels  seen  for  control  groups  in  the  tests  of 
other  programs  serving  individuals  who  were  already  employed  at  study  entry  (presented  in 
Chapters  4 and  5).  Over  the  four-year  follow-up  period,  the  average  quarterly  employment  rate 
was  62  percent,  and  average  earnings  were  just  under  $9,000  per  year  for  control  group  mem- 
bers (Table  4.7).  Average  earnings  increased  relatively  steadily  over  the  follow-up  period, 
without  a corresponding  increase  in  employment  — which  may  indicate  that  there  was  some 
advancement  over  time  in  the  control  group  (Appendix  Table  B.13).  While  most  of  the  control 
group  received  at  least  one  TANF  payment  in  the  first  follow-up  year,  in  the  fourth  year  of 
follow-up,  less  than  30  percent  of  the  control  group  received  at  least  one  TANF  payment 
(Appendix  Table  B.13). 

Employment  and  Employment  Retention 

Neither  the  Training  Focused  ERA  program  nor  the  Work  Plus  ERA  program  was  able 
to  produce  improvements  in  any  of  the  main  measures  of  employment  or  employment  retention 
in  the  cumulative  four-year  follow-up  period,  compared  with  the  levels  seen  in  the  control 
group  (Table  4. 7). 23  The  Training  Focused  ERA  program  decreased  employment  retention  in 
the  first  year  of  follow-up,  but  those  effects  disappeared  by  Year  2 (Appendix  Table  B.13).  This 
negative  effect  is  likely  a result  of  the  policy  that  allowed  Training  Focused  program  group 
members  to  reduce  their  weekly  work  hours  in  order  to  participate  in  education  and  training. 
Due  to  this  policy,  it  is  likely  that  some  program  group  members  substituted  training  for  work 
hours.  The  program  had  no  effects  on  employment  or  employment  retention  after  the  first  year 
of  follow-up. 

Earnings  and  Advancement 

There  is  no  evidence  that  either  the  Work  Plus  ERA  program  or  the  Training  Focused 
ERA  program  led  to  earnings  or  advancement  gains  in  the  four-year  follow-up  period,  beyond 


2'The  Work  Plus  ERA  program  did  have  a few  scattered  statistically  significant  impacts  on  the  measures 
shown  in  Appendix  Table  B.14.  Given  the  large  number  of  outcomes  shown  in  this  table,  however,  some 
impacts  may  be  statistically  significant  by  chance  and  may  not  be  due  to  any  real  effect  of  the  program.  Given 
that  there  are  no  statistically  significant  impacts  on  any  of  the  primary  measures  of  employment  retention  or 
advancement  for  the  Work  Plus  ERA  program,  these  other  impacts  should  be  viewed  with  caution. 
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what  was  achieved  in  the  control  group.  As  discussed  above,  among  sample  members  who  did 
not  have  a GED  certificate  or  high  school  diploma  at  the  time  they  entered  the  study,  the  Work 
Plus  and  Training  Focused  ERA  programs  both  produced  impacts  on  the  percentage  of  program 
group  members  who  participated  in  adult  basic  education  or  GED  preparation  classes  (Appen- 
dix Table  B.16).  Despite  these  increases,  the  programs  did  not  lead  to  advancement  gains  for  the 
full  sample  or  for  these  subgroups  (Appendix  Table  B.  17). 

Public  Assistance  and  Average  Annual  Income 

The  Training  Focused  ERA  program  group  received  9 percent  more  ($175),  on  average, 
per  year  in  TANF  cash  assistance  than  the  control  group  (Table  4.7).  Program  group  members 
substituting  education  and  training  for  employment  — thereby  qualifying  for  greater  TANF 
benefits  — may  have  caused  this  difference.  There  are  no  other  statistically  significant  impacts 
on  the  amount  of  TANF  or  food  stamp  assistance  received  or  on  average  annual  income  for 
either  of  the  Riverside  Phase  2 ERA  program  groups,  relative  to  the  control  group. 

Riverside  Phase  2 ERA  Programs:  Summary  and  Conclusions 

The  Riverside  Phase  2 ERA  programs  were  created  in  response  to  the  problem  that 
many  working  welfare  recipients  were  unable  to  retain  their  jobs  or  move  up  to  better  jobs. 
Therefore,  these  programs  — which  were  voluntary  and  delivered  services  through  ongoing 
staff-client  relationships  — were  designed  to  increase  retention  and  advancement  through 
improved  access  to  education  and  training.  The  Riverside  Work  Plus  ERA  program  encouraged 
enrollees  to  meet  the  welfare  system’s  quid  pro  quo  participation  requirements  by  combining  at 
least  20  hours  of  employment  per  week  with  up  to  12  additional  hours  of  attendance  in  basic 
education,  postsecondary  education,  or  vocational  training.  The  Riverside  Training  Focused 
ERA  program  allowed  enrollees  to  substitute  additional  hours  in  school  or  training  for  hours  on 
the  job  or  to  forgo  employment  temporarily  and  instead  participate  full  time  in  approved  skill- 
building activities.  The  Work  Plus  ERA  program  was  operated  by  DPSS,  while  the  Training 
Focused  ERA  program  was  operated  by  the  county  workforce  development  agency.  DPSS  also 
operated  the  Work  Focused  program  — a limited-services  program  that  was  available  to 
members  of  the  control  group;  this  program  made  available,  on  request,  staff  assistance  with  job 
retention  issues  and  payments  to  defray  enrollees’  child  care,  transportation,  and  other  work- 
related  expenses. 

Both  the  Work  Plus  and  the  Training  Focused  ERA  staff  intensively  marketed  the  ERA 
programs  and  recruited  program  group  members;  thereafter,  staff  contacted  engaged  group 
members  monthly  to  verify  employment  hours,  arrange  support  services,  and  monitor  partici- 
pants’ progress  in  their  activities.  Riverside  Phase  2 ERA  staff  developed  client-specific 
employment  plans  and  encouraged  participation  in  education  and  training  programs.  However, 
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due  to  individuals’  employment  instability,  staff  in  both  programs  spent  more  time  on  employ- 
ment retention  activities  than  they  had  anticipated. 

Over  half  of  program  group  members  in  both  Riverside  Phase  2 ERA  programs  had 
contact  with  ERA  staff  or  staff  from  an  employment  program;  less  than  a quarter  had  had  recent 
contact  with  staff  as  of  the  12-month  survey.  Around  40  percent  of  program  group  members  in 
both  programs  participated  in  education  or  training  in  the  first  12  months  after  study  entry,  but 
few  had  received  a new  degree  or  certificate  by  the  end  of  this  period. 

Almost  one-third  of  the  control  group  participated  in  an  education  or  training  activity 
during  the  first  year  following  random  assignment.  The  Training  Focused  program  increased 
the  percentage  of  program  group  members,  relative  to  the  control  group,  who  participated  in 
education  or  training  during  this  year,  but  the  increase  was  less  than  10  percentage  points  and  is 
almost  entirely  explained  by  an  increase  in  participation  in  adult  basic  education  (not  in  training 
or  college)  by  a subgroup  of  program  group  members  who  did  not  have  a high  school  diploma 
or  GED  certificate  at  the  time  of  study  entry.  The  Work  Plus  program  also  increased  participa- 
tion in  adult  basic  education  for  this  subgroup.  Few  program  group  members  in  this  subgroup 
(less  than  5 percent)  actually  received  a GED  certificate  within  a year  of  study  entry. 

The  Riverside  Phase  2 ERA  programs  were  implemented  largely  as  designed.  Howev- 
er, neither  the  Work  Plus  nor  the  Training  Focused  ERA  program  produced  any  statistically 
significant  impacts  for  the  full  sample  or  the  above  subgroup  on  the  main  measures  of  employ- 
ment, employment  retention,  earnings,  or  advancement  in  the  cumulative  four-year  follow-up 
period,  compared  with  the  levels  seen  in  the  control  group. 

There  are  limitations  to  what  the  evaluation  of  the  Riverside  Phase  2 ERA  test  can  say 
about  the  link  between  participation  in  education  and  training  and  job  advancement.  This  test 
examines  whether  or  not  strategies  that  are  meant  to  boost  participation  in  education  and 
training  can  lead  to  advancement.  Thus,  in  interpreting  the  test’s  results,  one  first  must  examine 
whether  the  strategies  did,  in  fact,  boost  such  participation  and  in  what  types  of  programs.  As 
noted  above,  increases  in  participation  in  education  and  training  were  not  large  for  the  full 
sample  and  were  driven  primarily  by  increases  among  non-high  school  graduates  who  did  have 
substantial  increases  in  participation  — but  only  participation  in  adult  basic  education  courses, 
and  the  increases  rarely  resulted  in  the  receipt  of  new  education  credentials.  Thus,  the  strategies 
boosted  participation  in  education  in  only  a limited  way  and  for  the  least-skilled  sample  mem- 
bers. 


Possible  reasons  for  the  programs’  limited  effects  on  participation  in  education  and 
training  are  suggested  by  field  research.  ERA  staff  reported  difficulty  in  convincing  many 
employed  single  parents  — especially  those  working  full  time  — to  cut  back  on  their  hours  of 
work  (and  thus  their  earnings)  or  on  time  devoted  to  family  in  order  to  attend  school  or  training. 
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They  also  reported  that  some  other  program  group  members,  when  they  left  employment, 
stopped  participating  in  education  or  training  or  chose  not  to  enroll  in  education  or  training 
programs,  opting  instead  to  look  for  work. 

Furthermore,  as  has  been  shown  in  prior  research,  the  type  of  education  participation 
that  was  increased  among  non-high  school  graduates  — adult  basic  education  classes  — may 
not  have  a large  and/or  immediate  payoff  in  terms  of  economic  benefits  unless  these  classes  are 
accompanied  or  followed  by  postsecondary  education  or  training.24  Annual  impacts  on  econom- 
ic outcomes  do  not  indicate  any  trend  toward  positive  effects  for  the  two  ERA  programs, 
however,  even  by  the  fourth  year  of  follow-up. 

Finally,  the  Riverside  Phase  2 ERA  test  was  not  a test  of  whether  or  not  participation  in 
education  and  training  can  lead  to  advancement.  Rather,  it  was  a test  of  whether  or  not  strategies 
that  are  meant  to  boost  participation  in  education  and  training  lead  to  advancement. 


24See  Bos  et  al.  (2001);  and  Mumane,  Willett,  and  Boudett  (1995). 
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Chapter  5 


Findings  for  ERA  Programs  Serving  Individuals  Who 
Were  Employed  and  Not  Receiving  TANF 

Introduction 

The  Employment  Retention  and  Advancement  (ERA)  programs  that  are  presented  in 
this  chapter  generally  began  providing  services  to  people  when  they  were  employed  and  were 
not  receiving  Temporary  Assistance  for  Needy  Families  (TANF).  This  category  includes  five 
programs:  the  Cleveland  ERA  program;  the  ERA  programs  in  Eugene  and  Medford,  Oregon; 
the  Riverside,  California,  Post-Assistance  Self-Sufficiency  (PASS)  program;  and  the  South 
Carolina  ERA  program.  Among  these  programs,  only  the  Riverside  PASS  ERA  program  led  to 
increased  employment  retention  and  advancement. 


Overview  of  Program  Designs  in  This  Group 

The  programs  in  this  chapter  prioritized  both  employment  retention  and  advancement, 
to  varying  degrees,  for  low-wage  workers.  Only  the  South  Carolina  ERA  program  explicitly 
included  the  additional  goal  of  placing  into  jobs  unemployed  individuals  who  had  left  TANF. 
The  target  populations  for  these  five  programs,  on  the  whole,  had  relatively  recent  work 
experience  when  services  were  first  provided  to  them.  While  they  were  not  receiving  TANF 
when  they  enrolled  in  the  program,  the  target  populations  included  both  former  beneficiaries  as 
well  as  some  who  had  never  received  TANF.1 

With  shared  goals  and  target  populations,  some  common  program  strategies  and 
structures  existed  across  these  five  programs.  Table  5.1  compares  a selection  of  features 
implemented  in  the  ERA  programs  serving  this  target  group  and  demonstrates  that  each 
program  consisted  of  a bundle  of  features.  Appendix  C provides  brief  summaries  of  each 
program  (Appendix  Boxes  C.l  through  C.5)  and  provides  details  on  program  administration 
(such  as  operation  dates  and  funding).  A fuller  description  of  the  model  designs  and  the 
programs’  implementation  is  provided  in  the  ERA  site-specific  interim  reports.2 

Participation  in  ERA  services  was  entirely  voluntary  in  all  these  programs.  Because  all 
were  serving  low-wage  workers  or  individuals  who  were  not  receiving  TANF,  it  was  not 


’in  Medford,  the  target  group  also  included  low- wage  workers  who  were  current  or  former  recipients  of 
food  stamps  or  state  child  care  subsidies. 

2For  complete  citations,  see  Appendix  E,  which  also  notes  where  the  reports  can  be  found  on  the  Web. 
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The  Employment  Retention  and  Advancement  Project 
Table  5.1 


Comparison  of  Program  Features  Across  ERA  Programs  Targeting  Individuals 
Employed  and  Not  Receiving  TANF 


Program  Feature 

O' 

p 

p 

p 

$ 

cs?  r& 

$ / 

f # 

> / 

— s 

A-  <>> 

Staffing 

Staff  positions  - team-based  (vs.  solo) 

V 

Staff  performance  management  (standards;  incentives) 

✓ 

Flexible  staff  hours  and/or  locations 

V 

V 

Caseloads  - low 

V 

V 

V 

Staff/client  engagmenet 

Participation  requirments 

TANF  participation  requirements 

ERA  participation  requirements  beyond  TANF  ones 

Initial  outreach  - intensive 

V 

V 

V 

V 

V 

Initial  outreach  - financial  participation  incentives 

V 

Program  flow  - client  assessment 
and  individualized  service  package 

V 

V 

V 

V 

V 

Ongoing  - develop/maintain  an  employment  plan 

V 

V 

V 

V 

Initial  job  preparation  and  placement  services 

V 

Retention  services 

Reemployment  services 

V 

V 

V 

Financial  work  incentives 

V 

Emergency  financial  assistance 

V 

V 

Connections  to  other  services  (work  supports,  social  services) 

V 

V 

V 

V 

Staff  counseling  on  job-related  issues 

V 

V 

V 

V 

Advancement  services 

Supported  advancement  through  job  change 

Education  and  training  referrals  and/or  incentives 

V 

>/ 

V 

V 

Staff  counseling  on  job-related  issues 

V 

V 

V 

Employer  linkages 

V 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 

NOTES:  The  program  features  are  defined  in  Chapter  2. 

Check  marks  indicate  that  the  feature  is  present  in  the  ERA  program. 

The  models  and  their  implementation  often  evolved  over  the  study  period.  This  table  presents  the  features 
experienced  by  the  majority  of  the  study  participants  for  the  greatest  extent  of  time. 
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possible  to  mandate  participation  in  program  services.  Program  administrators  often  identified 
eligible  individuals  through  state  welfare  management  information  systems  that  recorded  TANF 
case  closures,  and  intensive  outreach  and  marketing  strategies  were  enlisted  in  all  programs.3 

Ongoing  relationships  between  clients  and  program  staff  were  central  in  all  the 
programs  described  in  this  chapter.  ERA  program  staff  often  provided  flexibility  to  clients,  in 
tenns  of  meeting  locations  and  times,  in  order  to  allow  for  frequent  contact.  Among  the  ERA 
program  features,  retention  and  advancement  services  dominated  the  programs  in  this  group. 
Typically,  retention  activities  included  reemployment  services  and  staff  coaching  on  job-related 
issues,  and  advancement  services  included  career  counseling  and  education  and  training 
referrals. 

Table  5.2  shows  that  sample  members  in  the  target  group  described  in  this  chapter  also 
shared  similar  demographic  characteristics  at  baseline,  or  the  time  of  random  assignment.  For 
example,  in  addition  to  being  employed  when  they  enrolled  in  the  program,  sample  members  in 
this  target  group  were  likely  to  have  an  education  credential,  with  between  56  percent  and  81 
percent  having  at  least  a high  school  diploma  or  a General  Educational  Development  (GED) 
certificate.  This  is  a higher  overall  percentage  than  found  for  the  other  two  target  groups, 
described  in  Chapters  3 and  4. 

With  their  common  focus  on  individuals  who  were  not  receiving  TANF  and  who  were 
working  or  recently  employed,  the  programs  described  in  this  chapter  also  shared  a number  of 
implementation  challenges  in  providing  services  to  this  population.  Given  these  challenges, 
sites  were  not  always  able  to  operationalize  certain  program  elements.  The  following  sections 
present  the  services  that  staff  actually  provided,  not  the  program  features  as  they  were  designed. 


’South  Carolina  was  the  only  program  in  this  group  to  use  financial  incentives  to  encourage  participation. 
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The  Employment  Retention  and  Advancement  Project 
Table  5.2 


Selected  Charateristics  of  ERA  Sample  Members  at  the  Time  of  Random  Assignment: 

Employed  and  Not  Receiving  TANF 


Characteristic3 

Riverside 

PASS 

Cleveland 

Eugene 

Medford 

South 

Carolina 

Female  (%) 

90.0 

81.9 

92.5 

91.1 

98.1 

Average  age  (years) 

31.5 

32.7 

29.4 

31.3 

31.7 

Race/ethnicity  (%) 

Flispanic 

49.4 

5.7 

6.3 

4.6 

0.4 

Black,  non-Flispanic 

16.4 

56.4 

3.1 

1.4 

81.0 

White,  non-Flispanic 

31.6 

34.4 

84.8 

89.9 

17.9 

Other 

2.6 

3.4 

5.9 

4.1 

0.7 

Average  number  of  minor  children 

2.1 

1.3 

1.8 

1.9 

2.3 

Age  of  youngest  child  (%) 

2 or  under 

36.1 

30.9 

44.6 

34.9 

17.7 

3 to  5 

23.9 

22.1 

22.3 

24.0 

32.7 

6 or  over 

40.1 

47.0 

33.1 

41.2 

49.6 

Fligh  school  diploma/GED  certificate  or  higher  (%) 

57.3 

73.7 

80.5 

79.0 

56.0 

Total  prior  AFDC/TANF  receipt  (%) 

None 

4.8 

NA 

12.5 

37.4 

0.0 

Less  than  2 years 

51.6 

NA 

57.2 

47.3 

45.2 

2 years  or  more 

43.6 

NA 

30.4 

15.3 

54.8 

Living  in  public  or  subsidized  housing  (%) 

12.0 

16.0 

24.3 

24.6 

NA 

Employed  in  quarter  prior  to  random  assignment b(%) 

76.9 

93.1 

47.5 

77.2 

50.8 

Employed  in  quarter  of  random  assignmentb(%) 

74.5 

99.4 

89.6 

91.2 

50.3 

Employment  in  the  past  3 years  (%) 

Did  not  work 

4.4 

NA 

7.5 

0.9 

NA 

Less  than  6 months 

23.9 

NA 

15.0 

6.0 

NA 

7 to  12  months 

20.1 

NA 

19.8 

11.3 

NA 

1 3 to  24  months 

22.3 

NA 

29.3 

18.1 

NA 

More  than  24  months 

29.3 

NA 

28.5 

63.7 

NA 

As  of  random  assignment  the  number  of 
hours  worked  per  week  among  employed  (%) 

1-19  hours 

4.1 

0.1 

0.0 

11.5 

NA 

20-3 1 hours 

32.2 

8.4 

47.9 

34.7 

NA 

32  or  more  hours 

63.6 

91.5 

52.1 

53.8 

NA 

As  of  random  assignment  the  hourly  wage 
among  employed  (%) 

Less  than  $5.15 

0.1 

0.1 

0.5 

1.8 

NA 

$5.15  to  $6.99 

38.9 

3.9 

23.7 

20.0 

NA 

$7.00  to  $9.99 

50.2 

72.9 

66.9 

63.8 

NA 

$10.00  or  more 

10.8 

23.1 

8.9 

14.4 

NA 

Sample  size 

2,770 

697 

1,179 

1,164 

2,776 

(continued) 
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Table  5.2  (continued) 


SOURCES:  Calculations  from  ERA  baseline  forms,  automated  records,  and  administrative  data. 

NOTES:  NA  = not  available.  In  this  case,  the  data  for  these  measures  were  not  collected. 
“Statistics  include  both  program  and  control  group  members. 
bThis  information  is  based  on  unemployment  insurance  (UI)  records. 
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Cleveland  ERA  Findings 


Cleveland  ERA  at  a Glance 

• Targeted  low- wage  workers  in  the  long-term  nursing  care  industry 

• Offered  retention  services  at  the  work  site 

• Was  implemented  strongly,  but  participation  levels  were  affected  by  staff  difficulties 
navigating  workplace  rules  and  space  constraints 

• Was  compared  with  a control  group  that  had  no  special  on-site  retention  initiatives 

• Increased  participation  in  retention  and  advancement  services  but  had  only  modest 
levels  of  overall  participation 

• Did  not  increase  employment  retention  or  advancement 

Cleveland  ERA  Test:  Introduction 

The  Cleveland  ERA  program,  which  operated  between  2002  and  2005,  aimed  to 
increase  job  retention  among  low-wage,  entry-level  workers  in  the  long-tenn  nursing  care 
industry.  The  program  served  sample  members  for  as  long  as  they  remained  employed  at  the 
finn,  for  up  to  about  one  year. 

Origins  of  the  Test 

A common  challenge  for  programs  providing  retention  and  advancement  services  is  the 
difficulty  of  engaging  workers,  whose  job  and  family  responsibilities  often  leave  them  with 
limited  time  to  participate  in  a program.  The  driving  force  behind  the  Cleveland  ERA  program 
was  to  take  services  to  the  workers  at  their  place  of  employment  for  the  sake  of  convenience. 

The  Cleveland  ERA  program  targeted  low-wage  workers  in  long-tenn  nursing  care 
facilities,  an  industry  with  traditionally  high  turnover  rates.4  The  long-tenn  care  industry  has 
historically  had  serious  issues  with  worker  retention  and  has  taken  a number  of  steps  to  address 
it,  including  setting  up  Employee  Assistance  Programs  (typically  offering  employees 
confidential  evaluation,  treatment,  and  refenals  to  services  for  a range  of  problems,  such  as 
substance  abuse,  mental  health,  and  other  personal  problems)  or  programs  to  reward  and 


4“Tumover”  is  defined  as  the  number  of  worker  resignations  and  terminations  in  a given  period  divided  by 
the  total  workforce.  For  a review  and  discussion  of  turnover  rates  in  the  long-term  care  workforce,  see  Stone 
(2001). 
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recognize  employees  for  their  achievements.  Such  initiatives,  however,  are  not  consistently  in 
place  across  the  industry. 

The  Cleveland  ERA  program  was  a voluntary,  employer-based  retention  program  that 
offered  ongoing  staff-client  relationships,  weekly  information  sessions  covering  a variety  of 
topics,  and  supervisory  trainings  for  the  supervisors  of  low-wage  workers.  The  program’s  chief 
goal  was  to  help  individuals  retain  the  specific  jobs  that  they  held  as  of  entry  into  the  program, 
through  addressing  mainly  employee-level  factors  that  contribute  to  job  turnover.  On  the  whole, 
the  program  did  not  seek  to  fundamentally  change  employers’  work  environments,  workplace 
rules  and  practices,  or  general  ethos,  even  though  the  services  were  based  at  the  work  site. 
Rather,  the  program  focused  on  the  workers  themselves,  and  not  on  employer  or  industry 
practices  that  may  be  associated  with  employment  retention  or  turnover. 

The  program  was  operated  by  a community-based,  nonprofit  social  service 
organization.  Unlike  other  tests  in  the  ERA  project,  firms  (rather  than  individuals)  were 
randomly  assigned  either  to  have  ERA  services  provided  to  their  workers  or  to  continue  with 
their  standard  practices.  While  it  was  difficult  to  recruit  a large  number  of  firms  to  participate, 
44  firms  were  eventually  part  of  the  study,  and  21  long-term  nursing  facilities  and  one 
manufacturing  firm  were  randomly  placed  in  each  of  the  two  research  groups.5  The  ERA  study 
compares  the  outcomes  of  workers  at  the  firms  providing  ERA  services  (that  is,  the  program 
group  firms)  with  the  outcomes  of  workers  at  the  firms  that  did  not  provide  these  services  (that 
is,  the  control  group  firms).6 

Labor  Market  Context 

For  most  of  the  period  when  the  Cleveland  ERA  program  was  operating,  the  area’s 
labor  market  was  struggling.  Between  2000  and  2003,  the  unemployment  rate  for  the  Cleveland 
area  increased  from  roughly  3.9  percent  to  6.0  percent;  and  from  2000  to  2007  (the  end  of  the 
study  period),  the  total  number  of  jobs  declined  by  almost  80,000.  While  the  national 
unemployment  rate  began  to  fall  in  2004,  the  Cleveland  area  rates  did  not  decrease 
substantially.  In  2007,  the  Cleveland  area’s  unemployment  rate  was  6.0  percent,  compared  with 
the  national  rate  of  4. 6 percent. 


5There  are  several  reasons  why  the  recruitment  of  firms  was  difficult.  Many  of  the  approached  firms  did 
not  meet  the  study’s  main  eligibility  criterion:  having  at  least  15  recently  hired,  low- wage  employees.  In 
addition,  among  eligible  firms,  some  were  not  interested  in  participating  because  of  anticipated  difficulties  in 
implementing  the  services;  some  lacked  interest  in  addressing  retention  issues;  and  some  had  concerns  about 
having  to  provide  data  for  the  study  if  they  were  assigned  to  the  control  group. 

6An  examination  of  the  characteristics  of  study  sample  members  confirmed  that  the  demographic 
characteristics  of  those  in  the  program  group  firms  were  very  similar  to  the  characteristics  of  those  in  the 
control  group  firms  at  the  time  of  random  assignment. 
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Target  Population 

The  target  population  for  the  Cleveland  ERA  program  consisted  of  workers  who  earned 
less  than  $13  per  hour  and  had  been  hired  within  the  six  months  prior  to  random  assignment  at 
42  small  to  medium-size,  long-term  nursing  care  firms.  (Two  firms  that  were  brought  into  the 
study  toward  the  end  of  intake  were  in  the  manufacturing  sector.)  The  finns  involved  in  the 
study  were  required  to  employ  at  least  15  low-wage,  newly  hired  workers,  and  the  firms  had  to 
operate  in  the  Cleveland  metropolitan  area.  Among  the  participating  firms,  over  40  percent  had 
150  to  200  employees.  Most  of  these  finns  had  high  turnover  among  their  entry-level 
employees,  with  nearly  a third  having  a turnover  rate  of  50  percent  or  higher,  prior  to  study 
entry.  Table  5.2  shows  selected  characteristics,  as  of  study  entry,  of  the  individuals  in  these 
finns  who  made  up  the  Cleveland  ERA  sample. 

Significance  of  This  Test 

The  Cleveland  ERA  test  examines  whether  an  employer-based  retention  and 
advancement  program  — providing  services  at  the  work  site  through  ongoing  staff-client 
relationships,  peer  support  groups,  and  supervisory  trainings  — can  decrease  turnover  among 
low-wage  workers  in  the  long-tenn  care  industry,  compared  with  workers  in  similar  firms 
without  such  on-site  services.  Thus,  in  this  test,  unlike  the  other  tests  in  ERA,  the  focus  was  on 
maintaining  employment  in  the  job  held  as  of  study  entry,  rather  than  on  maintaining 
employment  in  general  (including  employment  that  might  be  at  a different  employer).  Chart  5.1 
provides  a succinct  presentation  of  important  differences  between  the  services  available  to 
program  group  and  control  group  members  in  the  Cleveland  ERA  test. 

Cleveland  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

A community-based  social  service  organization  with  experience  providing  employment 
services  in  the  Cleveland  area  developed  and  operated  the  Cleveland  ERA  program.7  The 
program  was  funded  through  a variety  of  public  and  private  sources,  including  county  funds  and 
private  donations  (through  United  Way),  and  with  funding  raised  by  the  community-based 
organization. 


7Random  assignment  occurred  in  stages:  eight  firms  were  randomly  assigned  in  September  2002;  14  firms 
were  randomly  assigned  in  November  2002;  eight  firms  were  randomly  assigned  in  February  2004;  and  14 
firms  were  randomly  assigned  in  May  2004. 
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The  Employment  Retention  and  Advancement  Project 
Chart  5.1 

Program-Control  Group  Differences: 


Cleveland  ERA  Test 

ERA  Group 

Control  Group 

Goals 

• Increase  retention  among  low-wage,  entry-level 
workers  in  the  long-term  nursing  care  industry 

Program 
resources  and 
structures 

• Operated  by  community-based,  social  service 
organization 

• Employer-based  delivery  of  services 

• Limited  and  inconsistent  employer- 
specific  initiatives  aimed  at  reducing 
turnover  among  low-wage  employees 
(such  as  Employee  Assistance  Plans  or 
programs  to  reward  and  recognize 
employees  for  achievements)2 

Staffing 

• Services  provided  by  generalist  staff 

• Staff  regularly  stationed  at  employers;  flexibility 
for  clients 

Staff-client 

engagement 

• Aggressive  marketing  and  outreach 

• Services  tailored  to  participants 

• Limited  ongoing  contact 

Retention 

services 

• Weekly  group,  life  skills  information  sessions 

• Provision  of  work  supports  and  social  service 
referrals,  as  needed 

• Help  with  problems  on  the  job 

Employer 

linkages 

• Supervisory  trainings  for  supervisors  of  entry- 
level  employees 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTE:  aEmployment-related  and  supportive  services  were  available  through  Workforce  Investment  Act 
providers,  One-Stop  Centers,  community  colleges,  adult  schools  and  other  education  providers,  and 
employment  and  training  organizations. 
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Staffing 

Services  in  the  Cleveland  ERA  program  were  provided  by  three  to  four  staff  (the 
number  varied  over  the  study  period)  who  provided  services  at  all  the  firms  in  the  program 
group.  All  staff  had  professional  backgrounds  in  social  work  and/or  experience  in  supportive 
services  and/or  crisis  intervention  programs.  On  average,  each  full-time  staff  member  was 
assigned  to  work  with  three  ERA  firms,  and  the  firms  worked  with  the  same  staff  person  over 
the  course  of  the  study.  Staff  had  a performance  goal  of  making  at  least  two  contacts  per  month 
with  each  program  group  member  at  each  firm  (a  goal  established  partway  through  the  study), 
although  there  were  no  financial  incentives  associated  with  achieving  this  target. 

ERA  staff  members  received  training  from  their  employer  (the  community-based 
organization)  as  well  as  from  a consultant  retained  by  MDRC  as  part  of  the  ERA  evaluation. 
The  MDRC-sponsored  training  focused  on  how  to  market  ERA  services  to  potential  employers 
and  individual  participants,  how  to  engage  participants,  and  how  to  deliver  retention  services  to 
participants. 

ERA  staff  were  stationed  at  employers’  work  sites  during  regularly  scheduled  “Office 
Hours”  and  provided  assistance  to  employees  on  work-related  issues  during  this  time.  Office 
Hours  were  scheduled  on  one  specific  day  per  week  in  each  firm,  for  a total  of  three  to  four 
hours  per  week.  In  order  to  meet  the  diverse  work  schedules  of  program  group  members,  ERA 
staff  also  provided  activities  during  the  evenings  and  during  third  (night)  shifts. 

Providing  both  paid  release  time  to  workers  to  meet  with  ERA  staff  and  physical  space 
for  meetings  was  not  a requirement  for  employers  during  the  early  stages  of  the  study,  and  this 
hindered  early  employee  participation  in  the  program.  During  later  stages  of  the  study,  firms 
were  required  to  provide  paid  release  time  to  all  participating  employees,  space  for  both 
individual  and  group  sessions,  and  assistance  in  scheduling  activities. 

Staff-Client  Engagement 

The  voluntary  nature  of  the  Cleveland  ERA  program  required  staff  to  use  aggressive 
marketing  and  recruitment  strategies,  including  initial  “intake”  meetings  to  engage  employees 
and  recruit  them  into  the  program.  There  were  no  financial  incentives  for  employees  to 
participate.  In  order  to  ensure  that  employees  had  an  opportunity  to  attend  an  intake  session, 
multiple  intake  sessions  were  held  over  the  course  of  a few  weeks  and  outside  traditional  work 
hours.  In  these  one-on-one  meetings,  ERA  staff  collected  background  information  on  the 
employee  and  explained  the  main  ERA  services. 

During  the  established  Office  Hours,  staff  provided  one-on-one  assistance  to  workers 
on  a range  of  factors  that  might  affect  retention,  including  issues  at  the  workplace  as  well  as 


166 


housing,  transportation,  and  child  care  matters.  Cleveland  ERA  staff  had  discretion  in  the  type 
of  assistance  that  they  provided  to  workers,  and  this  could  include  referrals  to  services  offered 
by  public  agencies  and  by  the  community. 

It  was  challenging  to  secure  ongoing  participation  from  employees,  given  their  work 
schedules.  Long-term  nursing  home  facilities  had  round-the-clock  shifts  that  required  staff  to  be 
on  call  for  patient  care  throughout  their  shifts.  In  addition,  given  the  physical  constraints  of 
nursing  homes,  it  was  often  difficult  to  secure  appropriate  space  for  meetings.  Over  time,  in  an 
effort  to  engage  more  employees,  ERA  staff  increasingly  sought  them  out  in  the  hallways  of  the 
nursing  home  facilities. 

Table  5.3  shows,  according  to  12-month  survey  data,  that  program  group  members, 
compared  with  control  group  members,  had  more  than  double  the  average  number  of  contacts 
with  staff  from  an  employment  program  in  the  first  12  months  following  random  assignment, 
with  about  seven  average  contacts  per  program  group  member.  Similarly,  program  group 
members  were  twice  as  likely  as  control  group  members  to  have  had  recent  contact  with  staff 
from  an  employment  program. 

Initial  Job  Preparation  and  Placement  Services 

Initial  job  preparation  and  placement  services  were  not  a priority  of  the  Cleveland  ERA 
program,  as  the  target  population  was  defined  as  low-wage  workers  newly  hired  in  long-term 
nursing  care  facilities.  Similarly,  given  the  program’s  focus  on  maintaining  employment  in  the 
jobs  that  sample  members  held  as  of  study  entry  — rather  than  on  finding  a job  at  a different 
employer  — the  Cleveland  ERA  program  did  not  include  any  job  search  assistance. 

Nonetheless,  the  12-month  survey  found  a higher  rate  of  participation  in  a job  search 
activity  by  program  group  members  than  control  group  members  (Table  5.3).  Moreover, 
employees  starting  out  in  the  firms  assigned  to  the  program  group  reported  that  they  were 
generally  less  satisfied  with  their  work  environment  at  the  one-year  mark  than  employees 
starting  out  in  the  control  group  firms.  This  lower  level  of  job  satisfaction  may  partially  explain 
why  program  group  members  were  participating  in  job  search  services  at  greater  rates  than 
control  group  members.  Although  data  are  not  available  to  test  this  hypothesis,  the  Cleveland 
ERA  program  may  have  raised  employees’  awareness  of  retention  issues  at  their  employers  and 
led  them  to  expect  more  from  their  supervisors.  This,  in  turn,  may  have  led  them  to  begin  to 
examine  employment  opportunities  outside  the  job  they  held  as  of  study  entry. 

Retention  Services 

Retention  services  were  a central  component  of  the  Cleveland  ERA  program.  Weekly 
group  sessions  led  by  ERA  staff,  called  “Lunch  and  Learns,”  provided  life  skills  education  in 
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The  Employment  Retention  and  Advancement  Project 
Table  5.3 

Summary  of  Impacts: 

Cleveland 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P -Value 

ImDlementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

52.4 

30.2 

22  2 *** 

0.000 

Average  number  of  contacts  with  staff 

6.8 

3.1 

3 g *** 

0.007 

Talked  with  staff  in  past  4 weeks  (%) 

16.6 

8.4 

8.2  ** 

0.035 

Retention  services 

Participated  in  a job  search  activity  (%) 

44.6 

34.2 

10.4  ** 

0.086 

Received  help  dealing  with  problems  on  the  job  (%) 

17.9 

7.7 

IQ  2 *** 

0.005 

Advancement  services 

Received  help  with  career  assessment  (%) 

9.0 

5.6 

3.5 

0.178 

Received  help  finding  a better  job  while  working  (%) 

6.3 

3.9 

2.4 

0.330 

Participated  in  an  education/training  activity3  (%) 

28.9 

37.3 

1 

0° 

* 

0.152 

Received  any  help  with  retention/advancement"  (%) 

38.5 

27.5 

11.0  ** 

0.025 

Sample  size  (total  = 485) 

260 

225 

Cumulative  economic  outcomes  (Years  1-3) 

Employment  measures 

Ever  employed  (%) 

99.1 

98.2 

0.9 

0.296 

Average  quarterly  employment  (%) 

83.3 

81.9 

1.3 

0.664 

Had  employment  spell  of  at  least  4 quarters  (%) 

88.9 

89.7 

-0.8 

0.712 

Length  of  longest  employment  spell,  in  quarters 

9.1 

9.0 

0.1 

0.849 

Length  of  longest  unemployment  spell,  in  quarters 

1.8 

1.9 

-0.2 

0.689 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

14,462 

14,566 

-105 

0.858 

Quarters  with  earnings  of  $3,500  or  more  (%) 

54.6 

54.3 

0.3 

0.920 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  3 (%) 

Not  employed  in  one  or  both  years 

14.2 

14.2 

-0.1 

0.986 

Earnings  decreased  by  more  than  $250 

35.3 

36.5 

-1.2 

0.752 

Earnings  changed  by  less  than  $250 

9.4 

7.1 

2.3 

0.433 

Earnings  increased  by  $250  or  more 

40.9 

41.5 

-0.6 

0.873 

Sample  size  (total  = 697) 

381 

316 

(continued) 
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Table  5.3  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative  records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language  (ESL) 
instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college  courses,  or 
vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job,  and 
addressing  personal  problems. 


four  broad  areas  — Workplace  Success,  Money  Matters,  Personal  Wellness,  and  Continuing 
Education  — and  covered  such  topics  as  problem  solving,  time  management,  goal  setting, 
conflicts  in  the  workplace,  credit  repair,  budgeting,  and  the  Earned  Income  Tax  Credit  (EITC). 

During  Office  Hours,  Cleveland  ERA  staff  also  provided  advice  on  a wide  range  of 
topics,  including  job-related  problems,  issues  with  supervisors,  referrals  to  services  in  the 
community,  and  access  to  supportive  services,  such  as  payments  for  the  employees’  work 
uniforms,  eyeglasses,  and  college  enrollment  fees.  The  12-month  survey  shows  that  18  percent 
of  program  group  members  received  help  dealing  with  problems  on  the  job.  While  this  rate  is 
more  than  double  the  rate  of  help  received  by  the  control  group,  it  is  low,  given  that  this 
population  is  known  to  experience  many  problems  on  the  job. 

Advancement  Services 

Job  advancement  services  were  not  a priority  of  the  Cleveland  ERA  program. 
Advancement-focused  discussions  between  Cleveland  ERA  program  staff  and  program  group 
members  rarely  occurred,  and  the  12-month  survey  shows  no  differences  between  the  program 
and  control  group  in  terms  of  their  likelihood  of  receiving  help  with  advancement. 

Employer  Linkages 

In  addition  to  Office  Hours,  the  Cleveland  ERA  program  included  training  for  the 
supervisors  of  entry-level  employees  in  the  ERA  firms.  The  supervisory  training  sought  to 
address  some  of  the  broader  factors  that  contribute  to  worker  turnover,  such  as  poor  supervisor- 
supervisee  relationships  and  conflict-ridden  work  environments.  These  voluntary  training 
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sessions  were  designed  to  help  supervisors  enhance  their  supervisory  skills  and  achieve  better 
relationships  with  entry-level  staff.  Conducted  by  ERA  staff  members,  the  training  consisted  of 
various  activities,  such  as  role-playing  to  help  the  supervisors  learn  new  skills  and  techniques 
and  to  gain  support  and  insight  from  other  supervisors  who  dealt  with  similar  workplace 
situations. 

Cleveland  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Cleveland  ERA  program  can  be  summarized  as  follows: 

• The  Cleveland  ERA  program  did  not  produce  any  changes  in  the  main 
measures  of  employment,  employment  retention,  earnings,  or  advancement, 
relative  to  the  control  group,  in  the  three-year  follow-up  period. 

• Data  on  TANF  or  food  stamp  receipt  were  not  collected,  as  most  sample 
members  did  not  have  a recent  connection  to  TANF.  Thus,  an  analysis  of  the 
program’s  effects  on  public  assistance  receipt  or  average  annual  income  is 
not  possible. 

Control  Group  Outcomes 

Average  earnings  and  average  quarterly  employment  for  control  group  members  in 
Cleveland  were  greater  than  the  levels  seen  in  any  of  the  other  ERA  control  groups  examined  in 
this  report.  The  average  quarterly  employment  rate  was  over  80  percent,  and  average  earnings 
were  about  $14,500  per  year  for  control  group  members  over  the  three-year  follow-up  period 
(Table  5.3).  Average  annual  earnings  and  average  quarterly  employment  both  decreased 
steadily  over  the  follow-up  period  (Appendix  Table  C.2). 

Employment  and  Employment  Retention 

While  the  Cleveland  ERA  program  could  have  produced  economic  impacts  if  it  had 
been  able  to  stop  or  slow  the  above-mentioned  decline  in  employment  over  time,  there  were  no 
changes  in  the  program  group  relative  to  the  control  group  in  any  of  the  main  measures  of 
employment  or  employment  retention  in  the  three-year  follow-up  period.  The  ERA  program  did 
produce  effects  on  job  retention  at  the  firms  where  program  group  members  were  employed  at 
the  time  of  random  assignment,  increasing  the  proportion  employed  at  that  firm  in  the  quarter 
after  random  assignment  by  5 percentage  points.  However,  this  effect  disappeared  by  the  next 
quarter  (not  shown). 


170 


Earnings  and  Advancement 

The  Cleveland  ERA  program  did  not  generate  increases  in  any  measures  of  earnings  or 
advancement,  compared  with  the  control  group  levels. 

Cleveland  ERA  Program:  Summary  and  Conclusions 

The  Cleveland  ERA  program  sought  to  increase  job  retention  among  newly  hired, 
entry-level  workers  in  the  long-term  nursing  care  industry.  A common  challenge  for  programs 
providing  retention  and  advancement  services  to  those  who  are  working  is  that  the  workers  have 
limited  time  to  participate  in  program  services.  In  response,  the  Cleveland  ERA  model  situated 
services  at  the  workplace,  making  it  more  convenient  for  workers  to  take  advantage  of  them. 
The  Cleveland  ERA  program  was  voluntary  and  was  operated  by  a community-based  nonprofit 
social  service  organization  that  offered  ongoing  staff-client  relationships,  weekly  information 
and  peer  support  sessions  covering  a variety  of  topics,  and  supervisory  trainings  for  the 
supervisors  of  low-wage  workers.  Although  no  formal  services  were  provided  to  the  control 
group,  these  individuals  could  access  services  offered  by  their  employer  or  in  the  community. 

Overall,  field  research  suggests  that  the  Cleveland  ERA  staff  used  aggressive  strategies 
to  engage  employees  and  recruit  them  into  the  program.  Securing  ongoing  participation  from 
employees  was  challenging,  however,  particularly  given  the  schedules  of  program  group 
members:  employees  in  long-term  nursing  care  facilities  have  round-the-clock  shifts  and  are 
always  on  call  for  patient  care. 

Over  half  the  program  group  members  had  contact  with  ERA  staff  or  staff  from  an 
employment  program,  but  less  than  one- fifth  had  had  recent  contact  with  staff  at  the  12-month 
follow-up  point.  More  than  one-third  reported  receiving  help  with  retention  or  advancement. 
Given  that  services  were  provided  at  the  workplace,  however,  participation  was  not  as  pervasive 
as  initially  expected. 

Compared  with  control  group  members,  program  group  members  had  more  frequent 
contact  and  more  recent  contacts  with  staff  from  an  employment  program,  so  it  appears  that  the 
program’s  employer-based  location  and/or  efforts  to  increase  ERA  staff-employee  contact  did 
pay  off  to  some  extent.  While  the  program  more  than  doubled  the  percentage  of  program  group 
members  who  received  help  dealing  with  problems  on  the  job,  relative  to  the  control  group,  the 
overall  receipt  rate  of  this  service  was  low,  given  the  emphasis  on  it  in  the  program  design. 

The  Cleveland  program  did  not  generate  any  improvements  in  employment, 
employment  retention,  earnings,  or  advancement,  relative  to  the  control  group,  during  the 
cumulative  three-year  follow-up  period.  There  are  several  possible  explanations  for  why  the 
program  did  not  produce  impacts  on  these  economic  outcomes,  and  two  of  them  are  presented 
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here.  First,  even  with  services  located  at  the  employer,  sustained  engagement  of  workers  was 
difficult  to  achieve.  Although  many  of  the  problems  that  the  program  faced  in  engaging  workers 
in  services  were  specific  to  the  nursing  home  industry,  some  were  not.  Even  in  environments 
that  have  less  demand  for  patient  and  floor  coverage  than  in  nursing  homes,  workers  may  not 
have  the  time  to  leave  their  work  to  attend  activities.  Second,  the  underlying  assumption  of 
offering  services  to  employees  at  their  workplace,  and  encouraging  employees  to  work  through 
problems  at  their  jobs,  is  that  it  is  in  the  employees’  best  interest  to  stay  in  these  jobs.  This, 
however,  may  not  be  the  case:  other  jobs  may  have  more  advancement  potential  or  other 
advantages.  In  the  end,  although  the  Cleveland  ERA  program  was  implemented  largely  as 
designed,  it  was  not  effective  at  promoting  employment  retention  and  advancement. 
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Eugene  ERA  Findings 


Eugene  ERA  at  a Glance 

• Targeted  employed  individuals  who  had  left  TANF 

• Provided  individualized  services  through  staff-client  interactions  that  focused  on 
employment  retention  and  advancement 

• Was  operated  by  a welfare  agency  in  partnership  with  a community  college  that  ran  a 
One-Stop  Center 

• Was  compared  with  a control  group  that  had  access  to  employment-related  services  in 
the  community,  if  individuals  chose  to  pursue  them 

• Experienced  unanticipated  demands  to  provide  reemployment  services,  which  impeded 
staffs  ability  to  provide  career  counseling 

• Produced  some  effects  on  employment  rates  but  did  not  increase  employment  retention 
or  advancement 


Eugene  ERA  Test:  Introduction 

The  Eugene  ERA  program  aimed  to  improve  employment  retention  and  advancement 
among  low-wage  workers  who  had  previously  received  TANF  but  were  no  longer  receiving 
assistance.  The  study  enrolled  sample  members  from  June  2002  through  June  2004,  and  it 
offered  sample  members  services  for  one  year  after  enrollment. 

Origins  of  the  Test 

At  the  state  level  in  Oregon,  an  interest  in  employment  retention  and  advancement 
services  developed  as  program  administrators  recognized  that  their  “work-first”  welfare  policies, 
while  largely  successful  in  connecting  TANF  recipients  to  employment,  were  not  resulting  in 
long-tenn  economic  success  for  working  families.  State  policymakers  perceived  that  services 
were  needed  to  move  working  families  into  stable  employment  and  better  and  higher-paying  jobs. 
At  the  time,  there  were  a range  of  employment-related  and  supportive  services  available  through 
Workforce  Investment  Act  (WIA)  One-Stop  Centers,  community  colleges,  adult  schools  and 
other  education  providers,  and  employment  and  training  organizations.  These  services  focused  on 
helping  people  to  find  jobs.  Employment  retention  and  advancement  services  for  working 
individuals,  however,  were  largely  lacking.  Furthennore,  those  that  were  available  were  limited, 
and  individuals  had  to  identify  and  pursue  them  on  their  own. 

The  Eugene  ERA  program  was  designed  to  augment  the  services  available  for  working 
individuals  who  left  TANF,  by  providing  individualized  assistance,  referrals  to  services,  and 
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advancement-focused  career  counseling.  The  ERA  program  in  Eugene  was  also  designed  to 
capitalize  on  the  distinct  resources  — financial  and  human  — available  in  the  welfare  and 
workforce  development  systems,  through  joint  management  of  the  program  by  the  State  of 
Oregon  Department  of  Human  Services  (DHS)  and  a community  college  that  also  operated  the 
WIA  One-Stop  Center. 

Labor  Market  Context 

In  Eugene,  the  ERA  study  began  in  June  2002,  shortly  after  the  end  of  the  national  2001 
recession.  Even  prior  to  this  recession,  the  unemployment  rate  in  Eugene  was  higher  than  the 
national  rate  — 5.4  percent  in  Eugene  in  2000,  compared  with  4.0  percent  nationally.  In  2003, 
Eugene’s  rate  peaked  at  8.0  percent,  compared  with  the  national  peak  in  2003  of  6.0  percent.  As 
a result,  program  group  members  in  Eugene  initially  faced  challenging  conditions  for  securing 
and  retaining  employment  and  advancing  to  better  jobs.  Economic  conditions  did  improve  over 
the  study  period:  Eugene’s  unemployment  rate  fell  to  5.3  percent  in  2007.  At  the  time  of  the 
study,  the  minimum  wage  in  Oregon  was  one  of  the  highest  in  the  nation  at  $6.50  to  $6.90  per 
hour,  compared  with  a national  minimum  wage  of  $5. 1 5 per  hour. 

Target  Population 

The  Eugene  ERA  program  served  employed  individuals  who  had  left  TANF  up  to  one 
year  prior  to  when  the  program  began  and  who  were  working  at  least  20  hours  per  week. 
Individuals  entered  the  study  sample  when  they  submitted  an  employment  verification  form  to 
the  welfare  department  to  request  access  to  food  stamp  benefits  and  TANF’s  postemployment 
“transitional”  benefits  — child  care,  Medicaid,  and  transportation  assistance  that  can  be  made 
available  to  people  for  a period  of  time  after  they  leave  welfare  for  work.  Table  5.2  shows 
selected  characteristics  of  the  Eugene  ERA  sample  members  at  the  time  of  random  assignment. 
The  Eugene  sample  is  distinct  among  the  ERA  sites  for  its  relatively  high  proportion  of  people 
with  an  education  credential;  more  than  80  percent  of  sample  members  had  a high  school 
diploma  or  a GED  certificate  when  they  entered  the  study.  This  rate  is  among  the  highest  of  all 
the  ERA  program  samples. 

Significance  of  This  Test 

The  Eugene  ERA  test  examines  whether  a voluntary  program  that  provided  intensive 
retention  and  advancement  services  and  individualized  career  counseling  to  low-wage  workers, 
through  a collaboration  between  welfare  and  workforce  agencies,  can  improve  employment  and 
retention  for  those  who  leave  TANF,  compared  with  the  standard  postemployment  services 
available  in  the  community.  Chart  5.2  presents  the  important  differences  between  the  services 
available  to  program  group  and  control  group  members  in  the  Eugene  ERA  test. 
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The  Employment  Retention  and  Advancement  Project 
Chart  5.2 

Program-Control  Group  Differences: 


Eugene  ERA  Test 

ERA  Group 

Control  Group 

Goals 

• Help  former  TANF  recipients  retain  their  jobs 
and  advance  to  better  jobs  and  wages 

Program 
resources  and 
structures 

• Operated  by  welfare  office  partnership  with  a 
community  college/WIA  contractor 

• No  single  retention-  and  advancement- 
oriented  program  for  employed 
individuals  in  community;  however, 
could  access  various  employment-related 
community  services  should  they  choose 
to  pursue  thema 

Staffing 

• Two-person,  colocated  teams'  including  welfare 
agency  staff  and  community  college  career 
development  specialists 

• Staff  schedules  and  meeting  locations  flexible  for 
clients 

Staff-client 

engagement 

• Aggressive  marketing  and  outreach 

• Services  tailored  to  participants 

• Development  and  maintenance  of  employment 
plans 

• Monthly  contact  during  first  3 months;  declining 
frequency  thereafter 

• Required  to  pursue  services;  however, 
similar  reported  contact  rates  as  ERA 
group 

Retention 

services 

• Reemployment  services  (targeted  job  leads, 
resume  building,  application  help,  career 
counseling,  skills  assessments,  and  referrals  to 
training  dollars)  available  for  30  days  after  loss  of 
job 

• Provision  of  work  supports  and  social  service 
referrals 

Advancement 

services 

• Goals-focused  career  counseling 

• Limited  education  and  training  referrals 

• Required  to  pursue  education  and 
training  opportunities;  however,  similar 
reported  take-up  of  education  and 
training  as  ERA  group 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTE:  “Employment-related  and  supportive  services  were  available  through  Workforce  Investment  Act 
providers,  One-Stop  Centers,  community  colleges,  adult  schools  and  other  education  providers,  and 
employment  and  training  organizations. 
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Eugene  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

The  Eugene  ERA  program  was  funded  primarily  by  TANF  resources8  and  was  jointly 
managed  and  operated  by  the  State  of  Oregon  Department  of  Human  Services  (DHS)  and  a 
local  community  college  (Lane  Community  College)  that  also  ran  the  local  One-Stop  Center. 
The  Eugene  ERA  program  operated  in  three  DHS  branch  offices.  The  two  agencies  had  worked 
together  as  part  of  a local  consortium  focused  on  workforce  issues;  however,  this  was  the  first 
time  that  they  had  collaborated  in  operating  a service  program. 

Staffing 

Each  of  the  three  Eugene  ERA  program  offices  was  staffed  by  a two-person  team  of 
DHS  staff  and  community  college  career  development  specialists  who  were  both  located  at  the 
welfare  office.  DHS  staff  were  primarily  responsible  for  helping  participants  access  such 
support  services  as  food  stamps  and  “extended  benefits”  (primarily  child  care,  transportation 
assistance,  and  Medicaid  subsidies  that  are  available  for  a period  of  time  after  someone  leaves 
TANF  for  work).  DHS  staff  also  took  the  lead  in  directing  individuals  to  social  service 
providers  in  the  community.  Community  college  staff  focused  on  career  counseling  and  helping 
clients  move  up  career  ladders,  although  this  did  not  include  job  development  or  outreach  to 
employers.  These  staff  had  a background  in  counseling  as  well  as  knowledge  of  the  local  labor 
market  and  local  services  and  institutions.  The  ERA  program  in  Eugene  was  also  managed 
through  this  organizational  partnership,  with  each  agency  assigning  one  person  to  the 
management  team. 

The  ERA  staff  in  Eugene  were  held  to  specific  performance  benchmarks  (although 
without  financial  incentives  for  achieving  them)  in  terms  of  client  contact  and  take-up  of  ERA 
activities.  The  ERA  program  in  Eugene  provided  flexibility  when  accommodating  program 
group  members’  schedules,  including  phone  contacts  in  the  evening  and  meeting  locations 
outside  the  office.  Over  the  course  of  the  study,  however,  this  flexibility  was  eventually  limited 
by  high  caseloads  and  ERA  persomiel  policies,  which  restricted  the  ability  of  staff  to  make 
schedule  adjustments  and  to  conduct  home  visits. 

Prior  to  the  ERA  program,  neither  the  welfare  agency  nor  the  community  college  had 
experience  in  providing  career  advancement  or  job  retention  services  to  low-income  workers 
after  they  found  jobs.  To  address  this  issue,  MDR.C  sponsored  a series  of  staff  training  sessions 


8Some  support  for  the  program  was  also  provided  by  the  U.S.  Department  of  Labor. 
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that  focused  on  marketing  and  client  engagement,  working  with  employers,  and  techniques  to 
help  participants  advance. 

Because  the  program  group  members  were  required  to  check  in  with  program  staff  for 
supportive  services,  this  also  helped  ERA  staff  maintain  regular  contact  with  individuals  on 
their  caseload.  However,  as  field  research  suggests,  this  staffing  structure  presented  some 
challenges,  particularly  in  terms  of  working  through  cultural  differences  between  the  two 
partnering  organizations.  In  particular,  community  college  staff  had  a difficult  time  transitioning 
to  the  welfare  office  environment  — which  placed  a strong  emphasis  on  understanding  and 
adhering  to  welfare  rules  and  regulations  — and  adjusting  to  the  ERA  target  population,  which 
had  more  employment-related  barriers  than  other  populations  that  they  served.  These  factors 
strained  working  relationships  between  team  members. 

Staff-Client  Engagement 

A strong  ongoing  relationship  between  staff  and  clients  was  central  to  the  Eugene  ERA 
program.  Because  participation  in  ERA  was  voluntary,  the  ERA  staff  teams  in  Eugene  made  a 
concerted  effort  to  schedule  initial  intake  meetings  with  sample  members,  through  persistent 
phone  and  mail  contacts.  In  these  contacts,  Eugene  ERA  staff  emphasized  several  aspects  of  the 
program  to  “sell”  it  to  potential  participants,  such  as  the  staffs  focus  on  advancement  and  their 
ability  to  provide  career  assistance. 

Table  5.4  shows  that  program  group  members  reported  high  levels  of  contact  with  staff 
from  an  employment  program;  about  85  percent  reported  any  contact  with  such  a person  in  the 
12-month  survey.  However,  although  the  program  had  rigorous  outreach  and  marketing  efforts, 
the  survey  also  found  that  control  group  members  were  as  likely  as  their  program  group 
counterparts  to  have  had  contact  with  staff  from  an  employment  program  within  the  year.  Both 
groups  also  reported  a similar  number  of  average  contacts  and  were  similarly  likely  to  have  had 
recent  contacts. 

According  to  field  research,  ERA  staff  in  Eugene  focused  on  providing  individualized 
assistance  that  met  the  specific  needs  of  each  client.  Staff  developed  a strong  rapport  with 
clients,  encouraged  disclosure  about  personal  or  family  challenges,  sought  to  provide  immediate 
assistance,  and  maintained  an  open-door  policy.  Eugene  program  staff  sought  to  maintain  an 
overall  focus  on  job-related  and  retention  and  advancement  issues  while  simultaneously 
addressing  personal  and  family  challenges  through  referrals  to  community  resources. 

During  initial  meetings  (often  with  both  team  members),  ERA  teams  used  different  tools 
(including  goal  exercises,  job  search  strategy  assessments,  skills  statements,  and  worksheets)  to 
create  “retention  and  career  plans”  outlining  the  individual’s  need  for  services,  career  goals,  and 
any  need  for  vocational  assessments.  Subsequent  meetings  focused  on  conducting  any 
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The  Employment  Retention  and  Advancement  Project 
Table  5.4 

Summary  of  Impacts: 

Eugene 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

83.5 

85.6 

-2.2 

0.537 

Average  number  of  contacts  with  staff 

21.3 

17.6 

3.7 

0.142 

Talked  with  staff  in  past  4 weeks  (%) 

44.3 

39.4 

4.9 

0.311 

Retention  services 

Participated  in  a job  search  activity  (%) 

74.5 

67.3 

7.3  * 

0.094 

Received  help  dealing  with  problems  on  the  job  (%) 

9.6 

5.9 

3.6 

0.158 

Advancement  services 

Received  help  with  career  assessment  (%) 

17.8 

9.5 

8.3  ** 

0.014 

Received  help  finding  a better  job  while  working  (%) 

15.5 

6.8 

g g *** 

0.004 

Participated  in  an  education/training  activity3  (%) 

21.7 

25.1 

-3.4 

0.416 

Received  any  help  with  retention/advancement"  (%) 

38.0 

22.0 

16.0  *** 

0.000 

Sample  size  (total  = 440) 

220 

220 

Cumulative  economic  outcomes  f Years  1-3) 

Employment  measures 

Ever  employed  (%) 

95.9 

93.7 

2.2  * 

0.086 

Average  quarterly  employment  (%) 

61.2 

62.7 

-1.5 

0.431 

Had  employment  spell  of  at  least  4 quarters  (%) 

66.9 

68.8 

-1.9 

0.481 

Length  of  longest  employment  spell,  in  quarters 

6.4 

6.6 

-0.2 

0.401 

Length  of  longest  unemployment  spell,  in  quarters 

4.0 

3.8 

0.2 

0.396 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

8,191 

8,404 

-213 

0.601 

Quarters  with  earnings  of  $3,500  or  more  (%) 

27.3 

27.0 

0.3 

0.882 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  3 (%) 

Not  employed  in  one  or  both  years 

38.7 

32.0 

6.7  ** 

0.014 

Earnings  decreased  by  more  than  $250 

19.9 

23.9 

-4.0 

0.103 

Earnings  changed  by  less  than  $250 

6.8 

6.9 

-0.1 

0.954 

Earnings  increased  by  $250  or  more 

34.5 

37.1 

-2.6 

0.352 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

657 

674 

-17 

0.792 

Average  annual  food  stamps  received  ($) 

2,253 

2,252 

1 

0.992 

Average  annual  income3  ($) 

11,101 

11,330 

-229 

0.551 

Sample  size  (total  = 1,179) 

585 

594 

(continued) 
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Table  5.4  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

cThis  measure  represents  the  sum  of  Ui-reported  earnings,  TANF,  and  food  stamps. 


additional  assessments  and  helping  the  individual  move  toward  the  goals  that  they  had  outlined. 
ERA  staff  reviewed  the  plan  on  an  ongoing  basis,  updating  the  plan  accordingly. 

Frequent  contact  (with  a goal  of  at  least  monthly  contact  during  the  first  90  days  after 
enrollment)  was  also  emphasized  in  the  Eugene  ERA  program.  In  field  research  interviews, 
ERA  staff  reported  that  they  were  in  regular  contact  with  most  individuals  for  the  initial  three 
months.  Follow-up  after  this  period  was  less  consistent,  and  more  time  was  spent  on  individuals 
in  crisis  or  on  those  who  were  most  motivated  to  receive  services. 

Initial  Job  Preparation  and  Placement  Services 

Initial  job  preparation  and  placement  services  were  not  a priority  of  the  Eugene  ERA 
program,  as  the  target  population  was  employed  former  TANF  recipients  (who  therefore  were 
assumed  to  have  few  job  placement  needs). 

Retention  Sen/ices 

Employment  retention  was  a priority  of  the  Eugene  ERA  program.  Staff  provided  a 
number  of  services,  including  work  supports  and  social  service  referrals,  to  help  program  group 
members  stay  in  their  jobs. 

Despite  these  efforts,  job  loss  emerged  as  an  acute  issue  in  the  Eugene  ERA  program. 
By  the  time  of  their  initial  meeting  with  ERA  staff,  many  clients  had  already  lost  the  job  that 
they  had  reported  to  the  welfare  department  and  that  had  triggered  their  entry  into  the  study.  As 
a result,  ERA  staff  in  Eugene  devoted  substantial  attention  to  job  search  activities  with  “new” 
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clients.  The  teams  worked  closely  with  clients  on  finding  a new  job  within  30  days,  particularly 
as  program  eligibility  was  contingent  on  employment.9  Staff  often  tried  to  turn  job  loss  into  an 
opportunity  for  the  individual  to  move  toward  his  or  her  career  goals.  Services  included  targeted 
job  leads,  resume-building,  assistance  with  job  applications,  career  counseling,  skills 
assessments,  and  referrals  to  training  programs.  The  Eugene  ERA  program  also  offered  a series 
of  classes  that  focused  on  job  search  activities  that  incorporated  advancement-related  topics, 
such  as  career  ladders.  However,  while  program  group  members  reported  high  levels  of 
participation  in  job  search  activities  in  the  12-month  survey,  this  rate  of  participation  was  only 
slightly  higher  than  the  rate  reported  by  control  group  members  (Table  5.4). 

Staff  in  the  Eugene  ERA  program  also  provided  assistance  with  supportive  services  and 
transitional  benefits,  such  as  assistance  with  food  stamps,  transitional  child  care,  subsidized 
health  insurance,  gas  vouchers,  and  car  repair  money.  Program  group  members  were  also  able 
to  access  emergency  funds  for  housing  crises,  car  repairs,  and  other  urgent  needs. 

Finally,  ERA  staff  in  Eugene  made  efforts  to  address  problems  that  clients  reported  they 
were  experiencing  on  the  job.  ERA  staff  could  contact  employers,  with  permission  from  their 
client,  in  order  to  discuss  retention  issues;  however,  ERA  staff  reported  that  few  workers  were 
comfortable  with  these  contacts.  Overall,  the  12-month  survey  found  that  program  and  control 
group  members  had  similar  rates  of  receiving  help  to  deal  with  problems  on  the  job. 

Advancement  Sen/ices 

The  Eugene  ERA  program  included  work-based  and  human  capital  approaches  to 
advancement,  relying  largely  on  career  specialist  staff  to  provide  these  services.  As  indicated 
above,  Eugene  ERA  staff  began  working  on  advancement  from  the  first  time  they  met  with  a 
program  group  member.  Goals-focused  career  counseling  was  the  central  feature  of  ERA, 
designed  to  help  program  group  members  consider  their  long-term  goals  and  begin  moving 
forward  in  their  career  and  educational  paths.  The  ERA  career  specialists,  largely  responsible 
for  career  counseling,  were  well  suited  to  this  role,  and  interviews  with  DHS  staff  suggest  that 
they  brought  knowledge  of  the  local  labor  market,  of  Eugene’s  workforce  and  community 
college  systems,  and  their  career  counseling  experience.  ERA  staff  identified  job  leads  or 
suggested  how  the  individual  might  advance  on  the  job,  such  as  by  taking  on  additional  job 
responsibilities  or  hours  in  order  to  demonstrate  a strong  work  ethic  to  their  employer.  As 
described  above,  however,  unexpected  job  losses  redirected  resources  toward  reemployment 


9If  the  program  group  members  reapplied  for  TANF  when  they  lost  their  job,  they  were  directed  back  to 
the  state  TANF  program  and  became  ineligible  for  the  Eugene  ERA  services  until  they  found  a job.  When  the 
individuals  became  employed,  they  could  be  transferred  back  to  the  Eugene  ERA  staff  and  again  begin  to 
receive  retention  and  advancement  services. 
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activities  and  detracted  from  the  career  counseling  that  was  envisioned  in  the  Eugene  ERA 
program. 

Education  — particularly  vocational  and  skill  training  programs  — was  a component  of 
advancement  services,  although  it  was  used  selectively  and  was  based  on  individual  needs  and 
abilities.  Where  appropriate,  staff  counseled  program  group  members  about  the  need  for 
additional  education  or  skills  to  achieve  their  career  goals  and  helped  individuals  access 
appropriate  programs  and  financial  aid.  Overall,  in  the  12-month  survey,  program  group 
members  reported  slightly  higher  levels  of  participation  in  vocational  training,  compared  with 
control  group  members,  but  they  reported  similar  levels  of  participation  in  other  types  of 
education  (Appendix  Table  C.5). 

Based  on  the  12-month  survey,  program  group  members  were  more  likely  than  the 
control  group  to  receive  help  with  any  employment  retention  or  advancement  services.  About 
38  percent  of  the  program  group  reported  receiving  help  keeping  a job  or  advancing  to  a better 
job,  compared  with  about  22  percent  of  the  control  group  (Table  5.4). 

Employer  Linkages 

The  ERA  program  in  Eugene  was  not  designed  to  market  its  services  to  employers  or  to 
develop  career  advancement  opportunities  at  employers,  and  staff  did  not  undertake  efforts  in 
these  areas. 

Eugene  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Eugene  ERA  program  can  be  summarized  as  follows: 

• The  ERA  program  in  Eugene  did  not  affect  most  measures  of  employment 
retention  or  earnings  in  the  cumulative  three-year  follow-up  period.  ERA 
also  did  not  produce  gains  in  advancement,  compared  with  the  control  group. 

• ERA  did  not  generate  any  changes  in  the  amount  of  TANF  or  food  stamp 
assistance  received  or  changes  in  average  annual  income,  compared  with  the 
control  group  levels. 

Control  Group  Outcomes 

Average  quarterly  employment  rates  and  average  earnings  for  control  group  members 
in  the  Eugene  ERA  study  were  comparable  to  the  outcomes  for  control  groups  in  other 
programs  serving  individuals  who  were  already  employed  at  study  entry  (presented  in  Chapters 
4 and  5).  Average  quarterly  employment  was  63  percent,  and  average  earnings  were  almost 
$8,500  per  year  over  the  three- year  follow-up  period  (Table  5.4).  Average  annual  earnings  were 
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steady  over  this  period,  but  the  average  quarterly  employment  rate  decreased  over  time 
(Appendix  Table  C.6).  While  almost  40  percent  of  the  control  group  received  at  least  one 
TANF  payment  in  the  first  follow-up  year,  in  the  third  year  of  follow-up,  less  than  20  percent  of 
the  control  group  received  at  least  one  TANF  payment  (Appendix  Table  C.6). 

Employment  and  Employment  Retention 

The  ERA  program  in  Eugene  boosted  the  percentage  of  program  group  members  who 
were  ever  employed  in  the  three-year  follow-up  period,  by  about  2 percentage  points,  compared 
with  the  control  group.  ERA  did  not,  however,  generate  gains  in  the  main  measures  of 
employment  retention  in  the  cumulative  three-year  follow-up  period. 

There  were  reductions  in  employment  levels  late  in  the  follow-up  period  for  the 
program  group,  relative  to  the  control  group.  While  there  were  no  decreases  in  employment 
retention  for  the  program  group  in  the  first  two  years  of  follow-up,  in  the  final  year,  the  Eugene 
ERA  program  decreased  several  measures  of  employment  and  employment  retention  by  6 to  7 
percentage  points  (Appendix  Table  C.6).  Programs  can  produce  negative  impacts  on 
employment  if  they  have  the  effect  of  encouraging  TANF  receipt  or  participation  in  education 
or  training  at  the  expense  of  employment.  However,  the  ERA  program  in  Eugene  did  not 
include  services  that  would  directly  promote  these  outcomes,  and  the  effects  did  not  occur  early 
in  the  follow-up  period,  when  participants  were  most  likely  to  receive  services.  Other  analyses 
did  not  reveal  explanations  for  this  result.10 

Earnings  and  Advancement 

Measures  of  earnings  and  advancement  based  on  quarterly  UI  records  are  similar  for  the 
program  and  control  groups  (Table  5.4),  suggesting  that  the  Eugene  ERA  program  did  not 
produce  improvements  in  advancement.  In  fact,  there  is  evidence  that  the  control  group 
experienced  more  advancement  than  the  program  group  did  in  the  first  year  of  follow-up; 


10A  few  hypotheses  were  examined  to  see  whether  they  might  account  for  the  strange  pattern  of  results. 
One  factor  that  could  result  in  this  outcome  would  be  if  program  group  members  left  the  state  at  a higher  rate  in 
the  third  year  of  follow-up,  compared  with  the  control  group,  leading  to  a larger  decrease  in  the  number  of  UI 
records  received  for  the  program  group  compared  with  the  control  group.  In  the  third  year  of  follow-up, 
however,  receipt  of  TANF,  food  stamps,  and  unemployment  benefits  increased  among  the  program  group 
members,  as  the  rate  of  employment  decreased.  Although  not  statistically  significant,  the  pattern  of  increase  in 
these  benefit  levels  suggests  that  there  was  a real  decrease  in  employment  for  the  program  group  in  the  third 
year  of  follow-up  and  that  the  decrease  cannot  be  fully  explained  by  program  group  members’  leaving  the  state 
at  a higher  rate  than  control  group  members.  Another  factor  that  could  have  led  to  a large  decrease  in 
employment  levels  among  the  program  group  is  if  a large  employer  of  program  group  members  laid  off  a large 
proportion  of  its  labor  force.  An  analysis  of  employer  identification  numbers  available  through  UI  records  did 
not  support  this  hypothesis. 


182 


according  to  the  12-month  survey,  fewer  program  group  members  were  making  more  than  $9 
per  hour,  and  fewer  had  jobs  that  offered  retirement  plans,  compared  with  control  group 
members  (Appendix  Table  C.8). 

Public  Assistance  and  Average  Annual  Income 

The  ERA  program  in  Eugene  generated  no  changes  in  the  amount  of  TANF  or  food 
stamp  benefits  received  and  no  changes  in  average  annual  income,  relative  to  the  control  group, 
over  the  cumulative  three-year  follow-up  period  (Table  5.4). 

Eugene  ERA  Program:  Summary  and  Conclusions 

While  the  “work- first”  TANF  policies  in  Oregon  connected  individuals  to  employment, 
state  policymakers  perceived  that  additional  services  were  needed  to  move  working  families 
toward  stable  employment  and  better  and  higher-paying  jobs.  In  response,  the  Eugene  ERA 
program  was  designed  to  enhance  the  services  available  to  working  TANF  leavers,  and  the 
program  provided  individualized  assistance,  referrals  to  services,  and  advancement-focused 
career  counseling.  Participation  in  the  Eugene  ERA  program  was  voluntary,  and  the  program 
was  administered  by  a partnership  between  a welfare  agency  and  a community  college  that  also 
operated  the  local  One-Stop  Center.  Although  no  formal  services  were  provided  to  control 
group  members,  they  could  access  standard  community  services  that  were  available  to  low- 
wage  workers. 

Overall,  field  research  suggests  that  the  Eugene  ERA  program  enhanced  the  services 
available  to  working  participants.  Early  in  the  program  operations  period,  ERA  staff  achieved  a 
high  level  of  participant  engagement  in  services.  Focused  on  providing  individualized 
assistance,  the  ERA  staff  developed  a strong  rapport  with  participants  and  encouraged 
disclosure  about  personal  or  family  challenges  while  maintaining  a focus  on  job-related  and 
retention  and  advancement  issues.  ERA  staff  developed  career  plans  outlining  the  individual’s 
need  for  services  and  reviewed  them  on  an  ongoing  basis.  But  as  caseloads  and  workloads 
expanded  over  the  course  of  program  operations,  staff  found  themselves  unable  to  be  as 
proactive  as  they  had  been  at  the  outset.  When  job  losses  required  staff  to  redirect  their  efforts  to 
rapid  reemployment  and  detracted  from  the  intense  career  counseling  that  had  been  envisioned, 
staff  often  fried  to  turn  job  loss  into  an  opportunity  for  clients  to  move  toward  their  career  goals. 

Most  program  group  members  reported  contact  with  ERA  staff  or  staff  from  an 
employment  program,  and  nearly  half  of  those  had  been  in  recent  contact  with  staff  as  of  the  12- 
month  follow-up  point.  Three-quarters  of  program  group  members  participated  in  a job  search 
activity,  and  over  one-third  received  help  with  retention  or  advancement  in  the  first  12  months 
after  study  entry. 
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While  the  program  group  had  contact  with  staff  and  received  services  at  relatively  high 
levels,  control  group  members  received  services  at  similar  levels.  Control  group  members  were 
just  as  likely  as  their  program  group  counterparts  to  have  initial  and  ongoing  contacts  with  staff 
from  an  employment  program  within  the  year  following  study  entry.  In  addition,  although  the 
Eugene  ERA  program  did  increase  levels  of  participation  in  some  services  relative  to  the  control 
group,  the  increases  were  generally  less  than  10  percentage  points. 

Overall,  the  ERA  program  in  Eugene  did  not  increase  most  of  the  main  measures  of 
employment  retention,  earnings,  or  advancement  in  the  cumulative  three-year  follow-up  period. 
One  possible  reason  for  the  lack  of  economic  impacts  is  that  the  program  operated  in  an 
environment  where  extensive  services  appear  to  have  been  available,  as  reflected  in  the  high 
proportion  of  control  group  members  who  received  employment-related  assistance  from  non- 
ERA  service  providers.  A second  reason  is  that  sample  members  in  the  Eugene  ERA  program 
frequently  experienced  job  loss,  which  made  it  difficult  for  staff  to  focus  on  providing 
advancement  services. 

Through  a collaboration  between  managing  agencies,  the  Eugene  ERA  program 
provided  intensive  retention  and  advancement  services  and  individualized  career  counseling  to 
low-wage  workers.  Although  the  program  was  implemented  largely  as  designed,  it  did  not 
substantially  increase  participation  in  retention  or  advancement  services,  and,  probably  as  a 
result,  it  did  not  increase  employment  retention  or  advancement  outcomes  beyond  those 
achieved  through  the  control  group’s  participation  in  existing  services  in  the  community. 
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Medford  ERA  Findings 


Medford  ERA  at  a Glance 

• Targeted  working  individuals  — those  who  had  left  TANF  or  current  recipients  of  the 
Oregon  Food  Stamp  Employment  and  Training  Program  and/or  the  Employment 
Related  Day  Care  program 

• Provided  individualized  services  through  staff-client  interactions  that  focused  on  job 
retention  and  advancement 

• Was  operated  by  a partnership  between  the  welfare  agency  and  a nonprofit  employment 
agency 

• Compromised  service  delivery  by  funding  cuts  and  the  program’s  initial  focus  on  crisis 
intervention 

• Was  compared  with  a control  group  that  had  access  to  employment-related  services  in 
the  community,  if  individuals  choose  to  pursue  them 

• Minimally  increased  participation  in  some  key  services  — career  assessment  and 
education  and  training  — amid  low  overall  participation  rates 

• Did  not  increase  employment  retention  or  advancement 


Medford  ERA  Test:  Introduction 

The  Medford  ERA  program  aimed  to  promote  employment  retention  and  advancement 
among  employed  low-wage  workers  with  a past  connection  to  TANF  or  a current  connection  to 
other  public  benefits.  The  study  enrolled  sample  members  from  February  2002  through  April 
2004,  and  services  were  provided  from  February  2002  through  2005  for  up  to  one  year  after 
enrollment  in  the  program. 

Origins  of  the  Test 

Like  the  Eugene  ERA  program,  the  Medford  ERA  program  developed  as  state-level 
program  administrators  in  Oregon  recognized  that  their  “work-first”  welfare  policies,  while 
largely  successful  in  connecting  TANF  recipients  to  employment,  were  not  resulting  in  long- 
tenn  economic  success  for  working  families.  State  policymakers  perceived  that  services  were 
needed  to  move  working  families  into  stable  employment  and  better  and  higher-paying  jobs.  At 
the  time,  retention  and  advancement  services  for  working  individuals  were  limited  to  those 
provided  locally  through  Oregon’s  welfare  and  workforce  agencies.  In  Medford,  the  Oregon 
Department  of  Human  Services  (DHS)  provided  postemployment  services  — some  of  which 
helped  individuals  gain  access  to  support  services  — to  those  who  left  TANF.  But  individuals 
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had  to  be  proactive  in  seeking  out  these  services.  Employment-related  services  for  low-wage 
workers  also  were  available  through  One-Stop  Centers  or  in  the  community  at  large,  but,  again, 
their  receipt  was  contingent  on  client  motivation  and  knowledge  of  their  availability. 

The  Medford  ERA  program  was  viewed  as  the  logical  next  step  in  DHS ’s  welfare-to- 
work  efforts.  Through  ERA,  DHS  sought  to  help  former  welfare  clients  — as  well  as  current 
recipients  of  other  public  benefits  program  — not  just  to  retain  their  jobs  but  to  progress  to 
better-paying  jobs.  The  program  offered  retention  and  advancement  services  tailored  to  clients’ 
long-term  employment  goals  and  individual  circumstances.  Participation  in  services  was 
voluntary,  and  the  program  was  operated  by  a partnership  between  DHS  and  The  Job  Council,  a 
nonprofit  employment  service  provider  and  the  operator  of  a One-Stop  Career  Center. 

Labor  Market  Context 

The  Medford  ERA  program  operated  in  difficult  labor  market  conditions,  which  were 
worse  than  conditions  nationally.  In  2003,  Medford’s  unemployment  rate  peaked  at  7.7  percent, 
compared  with  the  national  rate  of  6.0  percent.  Medford’s  unemployment  rate  followed  national 
trends  and  declined,  to  5.7  percent  in  2007.  As  noted  above,  Oregon’s  minimum  wage  rate  at 
the  time  was  one  of  the  highest  in  the  nation:  $6.50  to  $6.90  per  hour,  compared  with  a national 
minimum  wage  of  $5.15  per  hour. 

Target  Population 

The  Medford  ERA  program  targeted  employed  individuals  who  were  former  recipients 
of  TANF  or  current  recipients  of  the  Oregon  Food  Stamp  Employment  and  Training  Program 
and/or  the  Employment  Related  Day  Care  program.  Potential  sample  members  were  identified 
via  lists  of  people  who  had  left  TANF  and  were  working  or  who  were  current  recipients  of  these 
other  benefits  and  were  working.  When  staff  contacted  individuals  on  the  lists,  they  explained 
the  ERA  program  and  asked  individuals  if  they  were  interested  in  potentially  participating. 
Those  who  were  interested  were  subsequently  randomly  assigned  to  a research  group.  Table  5.2 
shows  selected  characteristics  of  the  Medford  ERA  sample  members  at  the  time  the  study 
began. 


Significance  of  This  Test 

The  Medford  ERA  test  examines  whether  a voluntary  program  that  is  administered 
through  a welfare  agency  partnership  with  a nonprofit  organization  and  that  provides 
individualized  staff  assistance,  advancement-focused  career  counseling,  and  referrals  to 
postemployment  services  can  improve  employment  retention  and  advancement  outcomes  for 
former  TANF  recipients  and  other  low-wage  workers,  compared  with  outcomes  achieved 
through  standard  postemployment  services  in  the  community.  Chart  5.3  presents  important 
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differences  between  the  services  available  to  program  and  control  group  members  in  the 
Medford  ERA  test. 

Medford  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

The  Medford  ERA  program  operated  with  funding  primarily  from  TANF.11  Partway 
through  the  study  period,  the  program  experienced  a reduction  in  resources  due  to  TANF 
funding  cuts  in  Oregon.  As  a result,  staff  hours  and  pay  were  reduced,  and  management 
turnover  reduced  supervision  of  the  ERA  program  — all  adversely  affecting  staff  morale. 
Funding  for  supportive  services  also  diminished  over  the  course  of  the  study  period. 

The  program  was  jointly  run  by  DHS  and  The  Job  Council,  a nonprofit  employment 
service  provider  and  the  operator  of  a One-Stop  Career  Center.  DHS  and  The  Job  Council  had  a 
strong  collaborative  working  relationship  that  predated  ERA.  Through  this  partnership,  DHS 
intended  to  leverage  The  Job  Council’s  expertise  and  workforce  development  network,  as  well 
as  to  distance  the  ERA  program  from  the  welfare  agency. 

Staffing 

Medford  ERA  program  services  were  delivered  by  one  five-member  team  located  at 
The  Job  Council’s  administrative  offices.  This  team  comprised  a generalist  frontline  staff 
member,  a job  coach,  a job  counselor,  a job  developer,  and  a learning  specialist,  and  the  team 
was  supervised  by  a project  manager.  These  staff  were  initially  drawn  from  four  partner 
agencies:  DHS,  The  Job  Council,  the  Oregon  Employment  Department  (ED),  and  Rogue 
Community  College  (RCC).12  The  colocation  of  staff  and  services  facilitated  communication 
among  team  members. 

The  team  shared  the  program  caseload.  Because  program  group  members  did  not  have 
a specific  ERA  staff  member  assigned  to  them,  this  structure  gave  program  participants  the 
flexibility  of  receiving  assistance  from  any  staff  member.  Within  this  structure,  the  ERA  staff 
were  viewed  as  generalists  who  were  able  to  provide  a wide  range  of  services.  As  the  program 
progressed  and  more  clients  enrolled,  staff  began  to  specialize  in  their  respective  areas  of 
expertise.  For  example,  DHS  staff  addressed  issues  relating  to  accessing  supportive  services, 
and  the  job  counselor  concentrated  on  providing  job  retention  services.  To  provide  flexibility, 


1 1 Some  support  for  the  program  was  also  provided  by  the  U.S.  Department  of  Labor. 

12Midway  through  the  study  period,  the  ED  and  RCC  staff  withdrew  from  the  team  due  to  funding  cuts. 
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The  Employment  Retention  and  Advancement  Project 
Chart  5.3 

Program-Control  Group  Differences: 
Medford  ERA  Test 


ERA  Group 

Control  Group 

Goals 

• Help  low-wage  working  people  retain  their  jobs 
and  advance  to  better  jobs  and  wages 

Program 
resources  and 
structures 

• Operated  through  partnership  between  a welfare 
agency  and  a nonprofit  employment  service 
provider 

• No  single  retention-  and  advancement- 
oriented  program  for  employed 
individuals  in  community;  however, 
control  group  members  could  access 
various  employment-related  services  in 
the  community  should  they  choose  to 
pursue  them3 

Staffing 

• Five-member,  colocated  team:  generialist  front- 
line staff,  job  coach,  job  counselor,  job  developer, 
and  learning  specialist 

• Staff  schedules  provided  flexible  for  clients 

Staff-client 

engagement 

• Aggressive  marketing  and  outreach 

• Services  tailored  to  participants 

• Developed  and  maintained  employment  plans 

• Limited  ongoing  contacts. 

• Required  to  pursue  services;  however, 
similar  reported  contact  rates  as  ERA 
group 

Retention 

services 

• Partway  through  operations,  emphasis  redirected 
from  attending  to  crises  and  employment  barriers 
to  employment-focused  retention  activities 

• Job  search  assistance  (preparing  resume,  job 
applications,  identifying  job  leads) 

• Required  to  pursue  services;  however, 
reported  similar  rates  of  participation  in 
job  search  activities  as  ERA  group 

• Assistance  with  supportive  services  and 
transitional  benefits;  social  service  referrals  as 
needed 

• Discussed  problems  on  the  job 

• Required  to  pursue  services;  however, 
reported  similar  rates  of  receiving 
support  services  and  help  with  basic 
needs  as  ERA  group 

Advancement 

services 

• Counseling  for  work-based  advancement 

• Limited  services  for  advancement  through  job 
change 

• Education  and  training  referrals 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 

NOTE:  “Employment-related  and  supportive  services  were  available  through  Workforce  Investment  Act 
providers,  One-Stop  Centers,  community  colleges,  adult  schools  and  other  education  providers,  and 
employment  and  training  organizations. 
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three  of  the  five  ERA  team  members  worked  nonstandard  hours  in  the  early  morning,  in  the 
evening,  and/or  during  weekends. 

The  Medford  ERA  staff  were  selected  for  their  positions  based  more  on  personal 
qualifications  — including  familiarity  with  programs  serving  low-income  individuals  and 
approaches  to  work  with  the  target  population  — and  less  on  credentials  or  agency  seniority 
(though  all  members  of  the  team  had  a bachelor’s  or  a master’s  degree).  MDRC-sponsored 
sessions  gave  team  members  additional  training  in  the  areas  of  marketing  and  participant 
engagement,  working  with  employers,  and  techniques  to  help  participants  advance  in  their  jobs. 

Staff-Client  Engagement 

An  ongoing  staff-client  relationship  was  central  to  the  Medford  ERA  program.  Sample 
members  in  Medford  were  not  receiving  TANF,  and,  as  a result,  participation  in  ERA  was 
voluntary.  Medford  ERA  staff  took  a proactive  approach  to  engage  individuals  both  before  and 
after  their  first  meetings.  The  ERA  team  sought  to  meet  with  program  group  members  within  15 
days  of  their  enrollment  into  the  program. 

Table  5.5  shows  that  many  program  group  members  did  have  contact  with  employment 
program  staff  in  the  first  year  following  study  entry;  about  73  percent  reported  any  contact  with 
a staff  member  from  an  employment  program  in  the  12-month  survey  — a rate  about  11 
percentage  points  greater  than  the  percentage  who  reported  such  contact  in  the  control  group. 

The  ERA  program  services  and  referrals  were  driven  largely  by  individuals’  career 
interests  and  support  needs.  During  the  first  meeting  with  program  group  members,  Medford 
ERA  staff  spent  time  learning  about  individuals’  work  histories,  their  personal  situations,  and 
their  motivations  and  goals  for  participating  in  the  program.  This  first  meeting  culminated  in  the 
development  of  a Personal  Development  Plan,  delineating  the  next  steps  that  individuals  agreed 
to  take  in  order  to  realize  their  goals.  Subsequent  meetings  were  generally  conducted  one-on- 
one,  and  they  revisited  progress  on  previously  identified  action  steps. 

Frequent  contact  with  clients  was  a priority  in  the  Medford  ERA  program,  and  staff  had  a 
minimal  contact  goal  of  once  a month.  Staff  used  a variety  of  methods  to  keep  program  group 
members  engaged,  including  establishing  trust  and  rapport  to  encourage  participation,  making 
phone  calls  or  sending  letters  when  program  group  members  missed  meetings,  and  finding 
motivational  hooks  when  participation  started  to  wane.  Program  group  members  had,  on  average, 
about  ten  contacts  with  a staff  member  from  an  employment  program  in  the  first  year  — nearly 
four  more  contacts,  on  average,  than  control  group  members  had  (Table  5.5). 

Interviews  with  ERA  staff  indicated  that  contact  levels  with  clients  diminished  over 
time.  The  12-month  survey  data  show  that  rates  of  recent  contact  for  both  program  and  control 
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Table  5.5 

Summary  of  Impacts: 

Medford 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

73.1 

62.3 

10.8  ** 

0.039 

Average  number  of  contacts  with  staff 

10.1 

6.4 

3.8  ** 

0.023 

Talked  with  staff  in  past  4 weeks  (%) 

24.2 

24.5 

-0.3 

0.953 

Retention  services 

Participated  in  a job  search  activity  (%) 

46.4 

43.0 

3.4 

0.532 

Received  help  dealing  with  problems  on  the  job  (%) 

6.0 

3.9 

2.2 

0.372 

Advancement  services 

Received  help  with  career  assessment  (%) 

16.5 

7.0 

9.6  *** 

0.006 

Received  help  finding  a better  job  while  working  (%) 

10.9 

3.3 

~j  g *** 

0.007 

Participated  in  an  education/training  activity3  (%) 

30.3 

21.6 

8.7  * 

0.074 

Received  any  help  with  retention/advancement"  (%) 

24.8 

16.1 

8.8  ** 

0.049 

Sample  size  (total  = 345) 

167 

178 

Cumulative  economic  outcomes  f Years  1-3) 

Employment  stability  measures 

Ever  employed  (%) 

94.8 

97.9 

_3  j *** 

0.005 

Average  quarterly  employment  (%) 

71.5 

72.8 

-1.3 

0.457 

Had  employment  spell  of  at  least  4 quarters  (%) 

80.4 

80.1 

0.3 

0.885 

Length  of  longest  employment  spell,  in  quarters 

7.6 

7.8 

-0.1 

0.532 

Length  of  longest  unemployment  spell,  in  quarters 

3.0 

2.8 

0.2 

0.249 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

10,010 

10,487 

-476 

0.225 

Quarters  with  earnings  of  $3,500  or  more  (%) 

35.2 

36.7 

-1.4 

0.432 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  3 (%) 

Not  employed  in  one  or  both  years 

24.8 

23.8 

1.0 

0.695 

Earnings  decreased  by  more  than  $250 

23.2 

24.1 

-0.8 

0.739 

Earnings  changed  by  less  than  $250 

6.8 

7.7 

-0.9 

0.564 

Earnings  increased  by  $250  or  more 

45.2 

44.5 

0.8 

0.793 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

216 

199 

17 

0.651 

Average  annual  food  stamps  received  ($) 

1,906 

1,872 

34 

0.627 

Average  annual  income3  ($) 

12,132 

12,558 

-426 

0.257 

Sample  size  (total  = 1,164) 

590 

574 

(continued) 
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Table  5.5  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

cThis  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


group  members  were  low  (less  than  25  percent  of  sample  members  reported  talking  with  an 
employment  program  staff  person  within  the  previous  four  weeks),  and  there  was  no  difference 
in  these  rates  between  the  two  research  groups.  Staff  reported  that  this  was  due  to  difficulties  in 
motivating  clients  to  participate  and  also  to  a reduction  in  staff  levels. 

Initial  Job  Preparation  and  Placement  Services 

Initial  job  preparation  and  placement  services  were  not  a priority  of  the  Medford  ERA 
program,  as  the  target  population  consisted  of  employed  individuals  (who  were  assumed  to  have 
relatively  few  job  placement  needs). 

Retention  Services 

The  Medford  ERA  program  design  initially  called  for  staff  to  assist  clients  in  the 
retention  of  their  employment  situations  by  focusing  on  addressing  personal  crises  and 
removing  specific  barriers  to  work.  Advancement  efforts  were  to  be  made  after  job  retention 
had  been  achieved.  However,  after  an  initial  operational  period,  field  research  showed  that 
Medford  ERA  staff  had  redirected  their  attention  and  were  focusing  more  on  providing 
employment-focused  retention  and  advancement  activities  and  less  on  doing  crisis  intervention 
and  solving  personal  problems.  Overall,  according  to  the  12-month  survey,  program  group 
members  were  more  likely  to  receive  some  type  of  help  with  employment  retention  and 
advancement  than  control  group  members  were. 

Job  search  assistance  became  a major  component  in  the  program  when  job  loss 
emerged  as  an  acute  issue.  Some  program  group  members  who  came  in  for  the  first  meeting 
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with  the  Medford  ERA  team  had  already  lost  the  jobs  that  had  originally  placed  them  on  the 
ERA-eligibility  lists.  Program  staff  spent  considerable  time  working  with  unemployed 
individuals,  identifying  job  leads  and  providing  assistance  in  preparing  resumes  and 
applications.  ERA  staff  also  counseled  employed  clients  about  not  quitting  their  current  job 
before  they  had  a new  job.  According  to  the  12-month  survey,  a considerable  portion  of 
program  group  members  (46  percent)  reported  participating  in  a job  search  activity,  although  a 
similar  proportion  of  control  group  members  also  reported  that  they  participated  in  a job  search 
(Table  5.5). 

Medford  ERA  staff  also  provided  assistance  with  supportive  services  and  transitional 
benefits  to  encourage  employment  retention,  including  assistance  with  food  stamps,  transitional 
child  care,  subsidized  health  insurance,  gas  vouchers,  and  car  repair  money.  In  limited  circum- 
stances, program  group  members  were  also  able  to  access  emergency  funds  for  housing  crises, 
car  repairs,  and  other  urgent  needs.  Medford  ERA  staff  also  made  social  service  referrals,  such 
as  to  mental  health  and  substance  abuse  services,  for  those  program  group  members  who 
identified  these  needs.  Despite  these  efforts,  the  12-month  survey  results  do  not  show  high  rates 
of  receipt  of  support  services,  help  with  basic  needs,  or  help  with  personal  problems  that  made  it 
hard  to  keep  a job;  moreover,  these  rates  of  receipt  do  not  differ  between  the  program  and  the 
control  groups  (Table  5.5  and  Appendix  Table  C.9). 

Advancement  Services 

While  the  Medford  ERA  program  lacked  a focus  on  advancement  issues  initially,  over 
time  the  program  included  services  that  promoted  both  work-based  and  human  capital  job 
advancement.  ERA  staff  assisted  individuals  in  advancing  in  their  current  jobs  and/or  helping 
them  identify  better  jobs  in  the  labor  market.  The  Personal  Development  Plan  described  above 
that  detailed  concrete  steps  toward  career  goals  was  central  to  the  career  counseling  services 
provided  by  the  program.  While  a relatively  small  portion  of  program  group  members  (about  17 
percent)  reported  receiving  help  with  career  assessment  according  to  the  12-month  survey,  this 
was  more  than  twice  as  many  as  reported  it  in  the  control  group  (Table  5.5). 

As  appropriate,  program  group  members  were  also  coached  on  how  to  speak  with  their 
employers  about  asking  for  more  hours,  a pay  raise,  or  a promotion.  Program  group  members 
were  also  taught  about  the  workplace  behaviors  that  could  make  them  strong  candidates  for 
advancement  opportunities.  Those  who  were  interested  in  a new  job  were  provided  such 
assistance  in  making  this  change  as  job  listings  and  referrals.  While  a small  portion  of  program 
group  members  (11  percent)  reported  receiving  help  finding  a better  job  while  working  within 
the  first  year  after  random  assignment,  this  is  more  than  triple  the  rate  reported  by  the  control 
group  (Table  5.5). 
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The  Medford  ERA  program  also  encouraged  participation  in  training  and  education  as  a 
route  to  job  advancement.  Program  staff  helped  participants  identify  short-term  vocational 
training  programs  that  would  allow  them  to  gain  the  skills  needed  to  advance  at  their  current 
employer  or  to  obtain  a new  job  in  an  industry,  but  they  also  allowed  individuals  to  pursue  a 
GED  certificate  or  other  more  education-focused  activities.  The  Medford  ERA  program  limited 
funding  to  directly  pay  for  these  opportunities,  but  ERA  team  members  helped  individuals 
apply  for  financial  aid.  The  12-month  survey  shows  that  almost  one- third  of  the  program  group 
members  at  least  started  an  education  or  training  activity  within  the  year  after  random 
assignment  — a participation  rate  that  is  9 percentage  points  higher  than  the  rate  of  the  control 
group. 


Employer  Linkages 

A final  feature  of  the  Medford  ERA  program  was  staff  efforts  to  contact  employers  on 
behalf  of  their  clients  to  discuss  retention  and  advancement  issues.  This  goal  never  fully 
materialized,  due  to  the  ERA  staffs  lack  of  experience  working  with  employers  and  to  the 
program  group  members’  preference  that  staff  not  contact  their  employers.  According  to  field 
research,  staff  were  not  proactive  in  suggesting  that  they  contact  employers  but,  rather,  waited 
for  clients  to  ask  for  such  intervention.  ERA  staff  reported  in  interviews  that  participants  rarely 
asked  them  to  contact  employers,  and  staff  respected  the  wishes  of  their  clients  in  this  regard. 

Medford  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Medford  ERA  program  can  be  summarized  as  follows: 

• The  Medford  ERA  program  decreased  the  percentage  of  program  group 
members  who  were  ever  employed  in  the  three-year  follow-up  period,  but  it 
did  not  generate  changes  in  the  main  measures  of  employment  retention  or 
earnings  during  that  period  relative  to  the  control  group.  There  is  no  evidence 
that  ERA  produced  advancement  gains,  compared  with  the  level  of 
advancement  seen  in  the  control  group. 

• ERA  in  Medford  did  not  affect  either  the  amount  of  TANF  or  food  stamp 
assistance  received  or  participants’  average  annual  income. 

Control  Group  Outcomes 

Average  earnings  and  average  quarterly  employment  for  control  group  members  in 
Medford  were  higher  than  the  levels  seen  in  all  but  one  of  the  other  ERA  control  groups 
examined  in  this  report  — the  control  group  in  Cleveland.  Average  quarterly  employment  was 
over  70  percent,  and  average  earnings  were  almost  $10,500  per  year  for  control  group  members 
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over  the  three-year  follow-up  period  (Table  5.5).  Average  annual  earnings  remained  fairly 
steady,  as  average  quarterly  employment  decreased  steadily  over  the  follow-up  period 
(Appendix  Table  C.10).  Just  under  10  percent  of  the  control  group  received  at  least  one  TANF 
payment  in  each  of  the  three  years  of  follow-up  (Appendix  Table  C.10). 

Employment  and  Employment  Retention 

The  Medford  ERA  program  did  not  generate  any  increases  in  any  of  the  measures  of 
employment  retention,  relative  to  the  control  group,  in  the  cumulative  three-year  follow-up 
period  (Table  5.5).  However,  ERA  produced  a decrease  of  3 percentage  points  in  the  proportion 
of  program  group  members  who  were  ever  employed  in  the  three-year  follow-up  period, 
reducing  it  to  95  percent  from  a control  group  level  of  98  percent.  It  appears  that  this  decrease 
was  concentrated  in  the  first  year  of  follow-up  (Appendix  Table  C.10). 

Earnings  and  Advancement 

There  is  no  evidence  that  the  Medford  ERA  program  led  to  earnings  or  advancement 
gains,  compared  with  the  control  group  levels  (Table  5.5). 

Public  Assistance  and  Average  Annual  Income 

In  the  cumulative  three-year  follow-up  period,  the  ERA  program  in  Medford  had  no 
effect  on  the  amount  of  public  assistance  received  or  on  participants’  average  annual  income 
(Table  5.5). 

Economic  Impact  Findings,  by  Study  Cohort 

Despite  the  funding  interruption  that  occurred  midway  through  operations  in  the 
Medford  ERA  program,  there  are  no  statistically  significant  differences  in  impacts  when 
analyzed  by  study  cohort  (not  shown). 

Medford  ERA  Program:  Summary  and  Conclusions 

While  the  “work-first”  welfare  policies  operating  in  Oregon  at  the  time  of  the  ERA 
study  connected  individuals  to  employment,  state  policymakers  perceived  that  additional 
services  were  needed  to  move  working  families  into  stable  employment  and  better  and  higher- 
paying jobs.  The  Medford  ERA  program,  which  was  voluntary  for  clients,  was  designed  to  help 
former  TANF  recipients  and  current  recipients  of  other  public  benefits  retain  their  jobs  and 
progress  in  the  labor  market.  The  program  was  administered  by  a partnership  between  the 
welfare  agency  and  a nonprofit  organization,  with  services  provided  by  an  interagency  team. 
The  model  components  included  individualized  relationships  between  clients  and  staff, 
advancement-focused  career  counseling,  and  referrals  to  postemployment  services.  Control 
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group  members  could  access  retention  and  advancement  services  available  in  the  community 
through  welfare  offices,  One-Stop  Centers,  or  other  organizations.  However,  to  gain  access  to 
these  services,  control  group  members  had  to  know  what  to  ask  for  and  had  to  take  the  initiative 
to  seek  the  services. 

Field  research  suggests  that  staff  in  the  Medford  ERA  program  focused  on  providing 
individualized  assistance  in  order  to  meet  the  specific  needs  of  each  participant.  ERA  staff 
initially  focused  much  of  their  effort  on  intervening  to  help  participants  solve  personal  crises 
rather  than  on  providing  a broader  set  of  retention  and  advancement  services.  After  a series  of 
training  sessions,  staff  reoriented  their  approach  more  toward  retention  and  advancement 
services  and  further  away  from  crisis  intervention.  Staff  provided  support  services  and 
promoted  advancement  through  advancement  on  the  job,  changing  jobs,  or  participating  in 
education  and  training.  However,  participants  experienced  frequent  job  loss,  requiring  staff  to 
redirect  their  efforts  to  rapid  reemployment  and  detracting  from  the  career  counseling  that  was 
envisioned  for  the  program.  In  addition,  cuts  in  funding  and  staffing  in  the  partner  agencies 
limited  the  ability  of  ERA  staff  to  provide  the  full  array  of  services  throughout  the  period  of 
program  operations. 

Three-quarters  of  program  group  members  reported  having  contact  with  ERA  staff  or 
staff  from  an  employment  program  in  the  first  year  after  study  entry.  Y et  only  a quarter  of  the 
program  group  participated  in  any  retention  or  advancement  services  in  this  first  year.  Despite 
the  fact  that  personalized  career  counseling  services  were  central  to  the  Medford  model,  less 
than  one- fifth  of  program  group  members  reported  receiving  help  with  career  assessment.  Staff 
in  the  Medford  ERA  program  also  encouraged  education  and  training,  and  almost  one-third  of 
the  program  group  members  at  least  started  an  education  or  training  activity  within  the  year 
after  study  entry. 

Compared  with  control  group  members,  program  group  members  were  more  likely  to 
have  contact  and  to  have  more  frequent  contact  with  staff  from  an  employment  program.  The 
program  also  produced  some  increases  in  individuals’  levels  of  participation  in  some  key 
services,  such  as  career  assessment,  job  search  activities,  and  education  and  training,  but  the 
increases  were  less  than  10  percentage  points.  Field  research  and  survey  data  suggest  that  client 
contact  and  engagement  suffered  toward  the  end  of  program  operations  — perhaps  exacerbated 
by  funding-related  staffing  problems. 

The  Medford  ERA  program  did  not  lead  to  any  improvements  in  the  main  measures  of 
employment  retention,  earnings,  or  advancement  during  the  cumulative  three-year  follow-up 
period.  Overall,  it  appears  that  a combination  of  factors  — including  the  time  it  took  to  focus 
staff  on  retention  and  advancement  issues,  funding  losses  that  hampered  program  management 
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and  staffing,  and  higher-than-expected  job  loss  among  clients  — presented  challenges  to  the 
implementation  of  the  ERA  program. 

But  other  factors  may  have  also  contributed  to  the  lack  of  program  effects.  Like  the 
Eugene  ERA  program,  the  Medford  ERA  program  operated  in  an  environment  where  control 
group  members  received  high  levels  of  employment-related  assistance  from  existing  service 
providers.  In  addition,  given  that  all  those  enrolled  in  the  study  sample  — program  group  as 
well  as  control  group  members  — were  interested  in  receiving  retention  and  advancement 
services,  the  Medford  control  group  members  may  have  been  particularly  likely  to  seek  out 
available  community  services  on  their  own  initiative.  Both  of  these  factors  probably  made  it 
difficult  for  the  program  to  significantly  boost  the  participation  levels  of  the  program  group 
above  the  levels  of  the  control  group. 

In  its  implementation,  the  Medford  ERA  program  was  similar  to  the  ERA  program 
operated  in  Eugene.  The  Medford  ERA  program  is  thus  another  example  of  a program  that 
provided  intensive  retention  and  advancement  services  and  individualized  career  counseling  to 
low-wage  workers  via  collaborating  organizations.  The  Medford  ERA  program  did  not 
substantially  increase  participation  in  retention  or  advancement  services,  and,  probably  as  a 
result,  it  did  not  increase  employment  retention  or  advancement  outcomes  beyond  those 
achieved  through  the  control  group’s  participation  in  existing  services  in  the  community. 
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Riverside  Post-Assistance  Self-Sufficiency  (PASS)  ERA  Findings 


Riverside  PASS  ERA  at  a Glance 

• Targeted  employed  individuals  who  had  left  TANF 

• Provided  multifaceted  retention  and  advancement  services,  including  job  search 
activities,  career  development  services,  supportive  service  payments,  and  referrals  to 
education  and  training 

• Was  operated  by  a different  organization  in  each  location  — primarily,  community- 
based  organizations  and  a community  college 

• Was  compared  with  a control  group  that  had  access  to  limited,  ongoing  postemployment 
services  through  the  welfare  agency,  if  individuals  chose  to  pursue  them 

• Encountered  unexpected  demands  to  help  clients  with  reemployment  and  funding  issues 
that  caused  services  to  end  earlier  than  planned 

• Measured  increases  in  participation  not  large,  but  based  on  data  for  a cohort  that  had 
were  few  positive  economic  impacts 

• Increased  employment  retention,  earnings,  and  advancement  through  the  end  of  the 
follow-up  period 


Riverside  PASS  ERA  Test:  Introduction 

The  Post-Assistance  Self-Sufficiency  (PASS)  ERA  program  in  Riverside  County, 
California,  aimed  to  promote  job  retention  and  advancement  among  employed  individuals  who 
had  recently  left  TANF.  The  study  enrolled  sample  members  from  July  2002  through  June 
2003,  and  it  offered  program  group  members  services  for  up  to  12  months  after  they  enrolled  in 
the  program. 

Origins  of  This  Test 

The  Riverside  PASS  ERA  program  was  initiated  in  response  to  recognition  by  the 
Riverside  County  Department  of  Public  Social  Services  (DPSS)  that  services  to  help  working 
former  TANF  recipients  stay  employed  and  to  enhance  their  career  development  were  lacking. 
In  fact,  prior  to  program  development  for  ERA,  there  were  only  limited  postemployment 
services  for  those  leaving  TANF  in  Riverside  County. 

Designed  as  a voluntary  program  that  provided  multifaceted  postemployment  services 
and  built  on  staff-client  relationships,  the  Riverside  PASS  ERA  program  offered  job  search 
activities,  career  development  services,  referrals  to  education  and  training  slots,  life  skills 
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workshops,  arrangements  for  supportive  service  payments,  and  referrals  to  social  service 
programs.  Based  on  the  assumption  that  organizations  other  than  the  welfare  agency  would  be 
more  familiar  with  the  jobs  and  services  available  in  their  communities  and  that  those  who  were 
leaving  TANF  would  be  more  willing  to  work  with  organizations  other  than  the  welfare  agency, 
several  community-based  organizations  and  a community  college  were  chosen  to  provide 
Riverside  PASS  ERA  services. 

Labor  Market  Context 

The  Riverside  PASS  ERA  program  began  shortly  after  the  end  of  the  2001  recession. 
The  unemployment  rate  in  Riverside  County  followed  national  trends,  falling  to  5.1  percent  by 
2006.  The  number  of  jobs  in  the  Riverside  area  grew  every  year  during  this  period. 

Target  Population 

The  Riverside  PASS  ERA  program  targeted  employed  individuals  who  had  left  TANF. 
Individuals  were  eligible  for  random  assignment  for  the  ERA  study  if  they  were  (1)  employed 
and  (2)  ineligible  for  TANF  in  the  current  month  but  had  been  eligible  in  the  prior  month.13 
Table  5.2  shows  selected  characteristics  of  the  Riverside  PASS  ERA  sample  members  at  the 
time  the  study  began. 

Significance  of  This  Test 

The  Riverside  PASS  ERA  test  examines  whether  a voluntary  program  of 
postemployment  services  and  supportive  service  payments  — provided  through  ongoing 
relationships  between  clients  and  staff  in  different  nonprofit  sector  agencies  (including 
community-based  organizations  and  a community  college)  — can  result  in  better  employment 
retention  and  advancement  outcomes  for  former  TANF  recipients,  compared  with  less  intensive 
postemployment  services  provided  by  local  welfare  agency  staff.  Chart  5.4  presents  the  key 
differences  in  the  services  available  to  program  group  and  control  group  members  in  the 
Riverside  PASS  ERA  test. 

Riverside  PASS  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

The  Riverside  County  DPSS  developed  the  PASS  program  in  2001.  Five  service 
providers  (three  community-based  organizations  [CBOs],  one  community  college,  and  one  DPSS 
office)  were  selected  to  deliver  the  program  at  various  locations  in  Riverside  County.  (Each 

1 'The  individual  (not  necessarily  the  entire  TANF  case)  had  to  be  ineligible  in  the  current  month. 
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The  Employment  Retention  and  Advancement  Project 
Chart  5.4 


Program-Control  Group  Differences: 
Riverside  PASS  ERA  Test 


ERA  Group 

Control  Group 

Goals 

• Employment  retention  and  advancement  among 
employed  individuals  who  recently  left  TANF 

Program 
resources  and 
structures 

• Operated  primarily  by  community-based 
organizations  and  a community  college,  as  well  as 
one  welfare  agency  office 

• No  single  retention-  and  advancement- 
oriented  program  for  employed 
individuals  in  community;  however, 
could  access  various  employment- 
related  community  services  should  they 
choose  to  pursue  them3 

Staffing 

• Various  service  delivery  models,  including 
coordination  among  partnering  organizations 

Staff-client 

engagement 

• Aggressive  marketing  and  outreach 

• Services  tailored  to  participants 

• Required  to  pursue  services;  however, 
similar  reported  contact  rates  as  ERA 
group 

Retention 

services 

• Job  search  assistance  (one-on-one  job  search 
assistance,  resume  assistance,  and  providing  job 
leads)  following  job  loss 

• Required  to  pursue  services;  however, 
similar  reported  rates  of  participation  in 
job  search  activities  as  ERA  group 

• Assistance  with  supportive  services  and 
transitional  benefits;  social  service  referrals  as 
needed 

Advancement 

services 

• Education  and  training  referrals 

• Required  to  pursue  education  and 
training  opportunities;  however,  similar 
reported  take-up  of  education  and 
training  as  ERA  group 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTE:  “Employment-related  and  supportive  services  were  available  through  Workforce  Investment  Act 
providers,  One-Stop  Centers,  community  colleges,  adult  schools  and  other  education  providers,  and 
employment  and  training  organizations. 
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program  group  member  was  served  by  just  one  provider.)  Except  for  the  DPSS  site,  these 
contractors  were  expected  to  have  familiarity  with  the  jobs  and  services  available  in  their 
communities  as  well  as  experience  working  with  employed  fonner-TANF  recipients.  It  was  also 
expected  that  former  TANF  recipients  would  be  more  likely  to  work  with  such  private 
organizations  than  with  the  welfare  agency. 

To  pay  for  program  operations,  DPSS  allocated  state  TANF  funds  to  each  of  the  four 
contracted  service  providers,  and  it  paid  directly  for  the  DPSS  program.  While  funding  was 
consistent  for  the  majority  of  the  program  operations  period,  the  Riverside  PASS  ERA  program 
was  impacted  by  significant  budgetary  problems  late  in  the  study  period.  As  a result,  DPSS 
decided  to  end  the  contracts  with  ERA  providers  earlier  than  intended,  that  is,  in  December 
2003.  Program  group  members  could  continue  to  receive  services  from  local  DPSS  office  staff 
through  June  2004.  Evidence  from  program  fracking  data  suggest  that  this  cutback  in  funding 
did  not  reduce  the  extent  to  which  program  group  members  received  services. 

Staffing 

The  staffing  of  the  Riverside  PASS  ERA  program  varied  across  the  service  providers, 
ranging  from  one  to  five  dedicated  ERA  staff  per  provider.  While  none  of  the  providers  used  a 
team  structure,  staff  within  or  among  partnering  organizations  often  did  work  together.  One 
CBO  formally  partnered  with  two  other  CBOs  to  serve  program  group  members,  and  it  defined 
specialized  roles  for  each  agency,  including  separate  responsibilities  for  job  search  and  job 
development.  One  DPSS  staff  person  provided  overall  management  of  the  Riverside  PASS 
ERA  program. 

Providing  flexibility  and  convenience  in  the  provision  of  services  was  generally  not  a 
priority  in  the  program.  According  to  a time  study  of  Riverside  PASS  ERA  staff,  most  staff  met 
with  program  group  members  during  regular  work  hours  (rather  than  during  night  and  weekend 
shifts)  and  a majority  of  in-person  contacts  were  office  visits.  However,  a portion  of  in-person 
contacts  included  home  visits,  and  one  CBO  provider  did  equip  “mobile”  staff  with  laptop 
computers  and  cell  phones  so  they  could  more  easily  connect  with  program  group  members. 

While  Riverside  PASS  ERA  providers  were  familiar  with  their  local  communities,  the 
ERA  staff  were  largely  (although  not  entirely)  inexperienced  at  running  postemployment 
programs  for  former  TANF  recipients.  Staff  had  performance  standards  for  contacts  and  service 
provision,  and  staff  performance  was  monitored  through  a computerized  fracking  system 
created  by  DPSS.  ERA  staff  received  technical  assistance  and  feedback  if  they  did  not  meet 
these  standards. 
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Staff-Client  Engagement 

Ongoing  staff-client  relationships  were  central  to  the  Riverside  PASS  ERA  program. 
Sample  members  were  all  former  TANF  recipients,  and  participation  in  the  program  was 
voluntary.  All  five  of  the  Riverside  PASS  ERA  service  providers  made  a concerted  effort  to 
contact  program  group  members  through  a combination  of  flyers,  brochures,  and  phone  calls  to 
encourage  them  to  enroll  in  the  program.  The  ERA  service  providers  used  different  recruitment 
approaches  and  tools,  such  as  unsolicited  home  visits;  offering  services  and  activities  to  the 
children  of  potential  participants;  advertising  such  participation  incentives  as  vision  care,  movie 
passes,  and  diapers;  and  sponsoring  an  event  where  goods  and  services  were  provided  to 
potential  clients’  families. 

Despite  these  efforts,  engaging  program  group  members  was  challenging.  Program  staff 
initially  could  not  locate  some  sample  members  due  to  incomplete  or  out-of-date  contact 
information  received  from  DPSS.  Once  contact  was  made,  ERA  staff  struggled  to  convince 
program  group  members  to  participate,  as  many  staff  did  not  know  how  to  “sell”  the  ERA 
services.  Table  5.6  shows  that  while  more  than  half  of  the  program  group  reported  in  the  12- 
month  survey  that  they  had  had  contact  with  staff  from  an  employment  program  since  study 
entry,  the  rate  was  similar  for  the  control  group.  However,  both  the  participation  levels  and  the 
impacts  on  participation  for  the  Riverside  PASS  ERA  program  should  be  interpreted  with 
caution.  The  participation  data  presented  above  are  from  the  12-month  survey,  which  was 
administered  only  to  a subset  of  sample  members  from  an  early  cohort  of  the  study,  for  whom 
there  were  few  positive  economic  effects.  The  low  participation  levels  captured  by  the  survey 
may  not  reflect  the  participation  levels  at  the  time  that  the  program  was  producing  economic 
benefits  for  program  group  members.  This  caution  extends  to  all  the  participation  results 
discussed  for  this  ERA  program. 14 

Once  program  group  members  were  contacted,  the  staff  in  the  Riverside  PASS  ERA 
program  provided  — as  observed  in  field  research  — individualized  services  that  were  tailored  to 
program  group  members’  needs  and  interests.  Each  of  the  five  program  providers  developed  its 
own  approach,  variously  using  letters,  phone  calls,  and  home  visits  to  assess  program  group 
members  and  prepare  individual  career  development  plans.  For  example,  staff-client  interactions 
at  one  CBO  revolved  around  arranging  supportive  service  payments,  and  there  was  little  effort  to 
further  engage  clients  when  this  was  completed.  In  contrast,  at  another  CBO  provider,  ongoing 
participation  was  more  comprehensive  and  included  the  fonnulation  of  career  development  plans 


l4Analysis  of  data  from  the  program’s  management  information  system,  reported  in  the  site-specific  report 
on  the  Riverside  PASS  program  (cited  in  Appendix  E),  suggest  that  the  program  may  have  increased  the  use  of 
a broader  range  of  services.  These  data  are  not  used  in  this  report  because  they  only  capture  services  reported  to 
the  welfare  department. 
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The  Employment  Retention  and  Advancement  Project 
Table  5.6 

Summary  of  Impacts: 

Riverside  PASS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes  (earlv  cohort"  onlv) 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

52.2 

55.2 

-3.0 

0.664 

Average  number  of  contacts  with  staff 

9.5 

6.9 

2.7 

0.410 

Talked  with  staff  in  past  4 weeks  (%) 

22.4 

23.2 

-0.8 

0.899 

Retention  services 

Participated  in  a job  search  activity  (%) 

62.2 

53.3 

8.9 

0.190 

Received  help  dealing  with  problems  on  the  job  (%) 

6.7 

6.7 

0.0 

0.998 

Advancement  services 

Received  help  with  career  assessment  (%) 

17.9 

6.3 

11.6  ** 

0.013 

Received  help  finding  a better  job  while  working  (%) 

7.6 

5.7 

1.8 

0.606 

Participated  in  an  education/training  activity1’  (%) 

42.1 

39.9 

2.2 

0.749 

Received  any  help  with  retention/advaneementc  (%) 

24.4 

20.0 

4.4 

0.454 

Sample  size  (total  = 224) 

120 

104 

Cumulative  economic  outcomes  (Years  1-4) 

Employment  measures 

Ever  employed  (%) 

90.5 

88.8 

1.7 

0.136 

Average  quarterly  employment  (%) 

59.7 

56.3 

3 4 *** 

0.006 

Had  employment  spell  of  at  least  4 quarters  (%) 

71.9 

69.4 

2.4 

0.146 

Length  of  longest  employment  spell,  in  quarters 

8.1 

7.6 

0.6  *** 

0.006 

Length  of  longest  unemployment  spell,  in  quarters 

5.4 

6.0 

-0.6  *** 

0.002 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

9,711 

8,843 

868  *** 

0.006 

Quarters  with  earnings  of  $3,500  or  more  (%) 

33.5 

29.5 

4 i *** 

0.001 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

39.4 

43.0 

-3.6  ** 

0.045 

Earnings  decreased  by  more  than  $250 

17.2 

15.7 

1.5 

0.317 

Earnings  changed  by  less  than  $250 

5.0 

4.3 

0.7 

0.429 

Earnings  increased  by  $250  or  more 

38.1 

36.7 

1.4 

0.457 

Public  assistance  and  income 

Average  annual  TANF  received  ($) 

1,257 

1,289 

-33 

0.644 

Average  annual  food  stamps  received  ($) 

933 

978 

44 

0.340 

Average  annual  income0  ($) 

11,901 

11,110 

790  ** 

0.010 

Sample  size  (total  = 2,770) 

1,627 

1,143 

(continued) 
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Table  5.6  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

The  12-month  survey  was  fielded  within  the  early  cohort  of  sample  members  who  were  randomly 
assigned  from  July  to  December  2002. 

This  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

This  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

This  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


and  participation  in  job  search  activities,  education  and  training  classes,  and  life  skills 
workshops.  In  addition,  gasoline  vouchers,  ancillary  payments,  counseling,  and  referrals  to 
social  service  agencies  were  provided. 

Overall,  the  Riverside  PASS  ERA  program  had  difficulty  maintaining  ongoing 
participation.  According  to  case  file  reviews  conducted  six  months  after  random  assignment 
began,  only  about  three- fifths  of  the  program  group  members  were  found  to  have  participated  in 
ERA  activities  consistently.  Results  from  the  12-month  survey  are  similar:  less  than  a quarter  of 
the  program  group  members  reported  that  they  had  recently  talked  with  staff  from  an 
employment  program  (Table  5.6). 

Initial  Job  Preparation  and  Placement  Services 

Initial  job  preparation  and  placement  services  were  not  a priority  of  the  Riverside  PASS 
ERA  program,  as  the  target  population  included  employed  former  TANF  recipients  (who  were 
assumed  not  to  need  job  placement  assistance).  Job  search  services  in  the  context  of 
reemployment  and  advancement  are  discussed  below. 

Retention  Services 

Job  retention  was  a priority  of  the  Riverside  PASS  ERA  program.  A number  of  services 
— including  job  search  assistance,  work  supports,  and  social  service  referrals  — were  provided 
to  help  program  group  members  maintain  their  employment  and  to  prevent  their  return  to  the 
TANF  rolls. 
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Job  search  assistance,  in  particular,  was  a major  service  component  in  the  program  due 
to  participants’  need  for  reemployment.  By  the  time  ERA  staff  contacted  program  group 
members  following  random  assignment,  many  had  lost  or  left  the  initial  jobs  that  had 
established  their  eligibility  for  the  study.  While  job  search  services  varied  by  ERA  provider, 
these  services  included  one-on-one  job  search  assistance,  help  with  creating  or  updating  a 
resume,  and  providing  job  leads.  For  example,  field  research  suggests  that  the  community 
college  provider  permitted  program  group  members  to  use  the  college’s  Job  Resource  Center, 
which  contained  office  equipment,  job  search  tools,  and  job  leads  and  was  staffed  by  a full-time 
job  search  specialist. 

All  the  Riverside  PASS  ERA  agencies  also  provided  a variety  of  supportive  service 
assistance  to  program  group  members.  These  supports  included  assistance  payments  for  rent 
and  utilities,  transportation  services  (gasoline  vouchers  and  car  repair),  assessment  of  child  care 
needs  and  help  with  processing  payments,  food  assistance,  referrals  to  legal  services,  paying  for 
books  and  other  school  supplies,  and  parking  fees.  ERA  providers  referred  program  group 
members  to  social  service  agencies  to  address  issues  of  domestic  violence,  emotional  and 
mental  health,  and  substance  abuse  — if  individuals  disclosed  such  problems  to  staff. 

Advancement  Sen/ices 

While  advancement  in  the  labor  market  was  a goal  of  the  Riverside  PASS  ERA 
program,  advancement  services  represented  a relatively  small  service  component,  compared 
with  retention  services.  The  primary  advancement  activities  were  career  counseling  and 
education  and  training  referrals. 

Staff  at  all  the  Riverside  PASS  ERA  agencies  provided  some  form  of  career 
development  advice,  although  the  content  varied  across  providers.  For  example,  staff  at  several 
agencies  provided  career  counseling  as  part  of  the  participants’  career  development  plans.  Yet 
formal  client  assessments,  goal  setting,  and  the  identification  of  action  steps  and  regular 
monitoring  of  progress  were  not  common  practices.  Furthermore,  field  research  suggests  that 
ERA  staff  at  some  organizations  lacked  skills  and  experience  in  this  area.  While  a low 
percentage  of  program  group  members  reported  in  the  12-month  survey  that  they  had  received  a 
career  assessment  (18  percent),  this  was  nearly  three  times  the  rate  reported  by  control  group 
members  (Table  5.6). 

For  education  and  training  services  (including  educational  and  vocational  training, 
office  skills,  and  life  skills  workshops),  Riverside  PASS  ERA  providers  referred  clients  to 
partner  agencies.  One  CBO  provider  made  referrals  by  taking  advantage  of  its  linkages  with 
education  and  training  providers  in  the  community.  Another  provider  was  a community  college; 
its  referrals  emphasized  educational  activities  on  its  campus.  According  to  MDRC  field 
research,  most  education  and  training  referrals  were  made  at  program  group  members’  request, 
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and  some  were  included  as  part  of  a career  development  plan.  While  a considerable  number  of 
program  group  members  (42  percent)  reported  participating  in  an  education  or  training  activity 
according  to  the  12-month  survey,  this  rate  was  not  significantly  different  from  the  rate  reported 
by  control  group  members  (Table  5.6). 

The  Riverside  PASS  ERA  program  did  not  promote  changing  jobs  as  a strategy  for 
labor  force  advancement.  Most  job  search  services  were  focused  on  reconnecting  out-of-work 
individuals  with  employment,  with  less  emphasis  on  upgrading  to  jobs  with  better  pay,  hours,  or 
career  opportunities.  Less  than  10  percent  of  program  group  members  reported  receiving  help 
finding  a better  job  while  they  were  working  — a rate  that  is  comparable  to  the  control  group’s 
(Table  5.6). 

Employer  Linkages 

The  Riverside  PASS  ERA  program  did  not  include  formal  connections  with  local 
employers.  However,  job  developers  at  one  of  the  CBOs  providing  ERA  did  work  with 
community  employers  to  identify  jobs  that  had  career  ladders. 

Riverside  PASS  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  Riverside  PASS  ERA  program  can  be  summarized  as  follows: 

• The  Riverside  PASS  ERA  program  produced  increases  in  several  measures 
of  employment  retention  and  earnings.  In  addition,  there  are  some  indications 
that  ERA  may  have  led  to  gains  in  advancement,  compared  with  the  level 
seen  in  the  control  group. 

• The  economic  gains  produced  by  the  Riverside  PASS  ERA  program  were 
concentrated  among  those  who  entered  the  study  in  the  second  half  of  the 
enrollment  period  rather  than  people  who  entered  the  study  in  the  first  half  of 
the  enrollment  period.  For  the  later  cohort,  ERA  generated  increases  in 
hourly  wages  and  work  hours. 

• The  Riverside  PASS  ERA  program  did  not  affect  public  assistance  receipt, 
but  earnings  gains  helped  to  produce  increases  in  average  annual  income 
over  the  cumulative  four- year  follow-up  period. 

Control  Group  Outcomes 

Average  quarterly  employment  rates  and  average  earnings  of  control  group  members  in 
the  Riverside  PASS  ERA  study  were  similar  to  those  of  control  groups  in  the  other  programs 
that  served  individuals  who  were  already  employed  at  study  entry  (presented  in  Chapters  4 and 
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5).  Average  quarterly  employment  was  just  over  55  percent,  and  average  earnings  were  just 
under  $9,000  per  year  for  control  group  members  over  the  four-year  follow-up  period  (Table 
5.6).  Average  annual  earnings  increased  steadily  over  time  even  though  average  quarterly 
employment  decreased  slightly  over  the  follow-up  period;  this  may  indicate  that  there  was  some 
advancement  over  time  among  the  control  group  members  (Appendix  Table  C.14).  Despite  the 
fact  that  the  Riverside  PASS  ERA  sample  comprises  those  who  had  left  TANF,  almost  half  the 
control  group  received  at  least  one  TANF  payment  in  the  first  follow-up  year,  and  about  20 
percent  did  so  in  the  fourth  year  of  follow-up  (Appendix  Table  C.14). 15 

Employment  and  Employment  Retention 

The  Riverside  PASS  ERA  program  produced  gains,  relative  to  the  control  group,  in 
several  measures  of  employment  retention  in  the  cumulative  four-year  follow-up  period  (Table 
5.6).  The  program  increased  average  quarterly  employment  by  3.4  percentage  points,  or  by 
about  6 percent  of  the  control  group  level.  It  also  increased  the  average  length  of  the  longest 
employment  spell  by  0.6  quarter,  or  by  about  8 percent  of  the  control  group  average.  In 
addition,  the  program  increased  the  percentage  of  program  group  members  who  were  employed 
during  the  entire  follow-up  period  by  3.4  percentage  points,  or  by  almost  17  percent  of  the 
control  group  level  (Appendix  Table  C.15). 

There  is  evidence  that  the  Riverside  PASS  ERA  program  helped  program  group 
members’  reemployment  efforts  after  the  loss  of  employment.  In  addition  to  increasing  the 
length  of  employment  spells,  the  program  also  decreased  the  average  length  of  the  longest 
unemployment  spell  by  about  0.6  quarter,  or  by  almost  10  percent  of  the  control  group  average 
(Table  5.6).  It  appears  that  at  least  part  of  this  effect  was  due  to  the  program’s  helping  program 
group  members  become  reemployed  more  quickly  than  control  group  members  after  the  loss  of 
employment.16  Most  program  and  control  group  members  left  their  initial  jobs  (the  jobs  they 
held  at  the  time  of  random  assignment)  at  some  point  during  the  follow-up  period,  and  they  left 
these  jobs  at  a roughly  equivalent  rate.  Thus,  there  is  no  evidence  that  the  Riverside  PASS  ERA 
program  had  an  effect  on  retention  of  the  initial  job.  The  program  generated  its  early  increases 
in  employment  primarily  by  increasing  the  proportion  of  program  group  members  — relative  to 
control  group  members  — who  quickly  found  a subsequent  job. 


In  Riverside,  a person  who  leaves  a welfare  case  — even  if  still  getting  a child-only  grant  — is  defined 
as  a welfare  leaver.  Since  public  assistance  data  were  collected  by  case  numbers,  some  of  the  welfare  receipt 
reported  here  may  reflect  payments  to  child-only  cases. 

16Some  of  this  decrease  also  could  have  resulted  from  the  program’s  leading  some  people  who  would  have 
had  an  unemployment  spell  without  the  program  to  remain  continuously  employed  throughout  the  follow-up 
period. 
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In  addition  to  helping  people  regain  employment  after  loss  of  employment,  the  program 
appears  to  have  helped  individuals  who  were  not  employed  in  Ul-covered  jobs  at  the  time  of 
study  entry  move  into  Ul-covered  employment  more  quickly  than  they  would  have  without  the 
program.  This  is  important  because  Ul-covered  jobs  typically  pay  higher  wages  and  are  more 
likely  to  offer  fringe  benefits  and  other  mandatory  benefits,  such  as  Social  Security  and 
unemployment  benefits,  than  jobs  not  covered  by  the  UI  system.  The  Riverside  PASS  ERA 
program  decreased  by  0.4  quarter  the  average  number  of  quarters  that  elapsed  before  program 
group  members  started  their  first  employment  spell  in  a Ul-covered  job,  or  by  about  18  percent 
of  the  control  group  average  (Appendix  Table  C.15). 

Figure  5.1  shows  that  the  Riverside  PASS  ERA  program  began  having  an  effect  on 
employment  levels  at  the  end  of  Year  1 and  that  it  continued  to  increase  employment,  relative  to 
the  control  group,  through  the  first  half  of  Year  4.  During  that  time,  statistically  significant 
impacts  on  employment  ranged  from  4 to  5 percentage  points,  representing  increases  of  about  7 
to  8 percent  of  the  control  group’s  average  quarterly  employment  levels. 

Earnings  and  Advancement 

The  Riverside  PASS  ERA  program  increased  earnings  and  led  to  additional 
advancement,  compared  with  the  experiences  of  the  control  group  (Figure  5.1).  The  program 
produced  earnings  gains,  relative  to  the  control  group,  in  almost  every  quarter  in  the  four-year 
follow-up  period.  The  statistically  significant  gains  ranged  from  $175  to  $330  per  quarter, 
representing  about  8 percent  to  14  percent  increases  relative  to  the  control  group’s  quarterly 
earnings.  Some  of  the  earnings  increases  were  due  to  the  program’s  effects  on  employment 
retention.  However,  throughout  the  follow-up  period,  the  program’s  effect  on  earnings  was 
greater  (in  terms  of  percentage  gains  relative  to  control  group  levels)  and  more  persistent  than 
its  effect  on  quarterly  employment.  In  the  last  two  quarters  of  follow-up,  the  program  increased 
earnings  by  $275  and  $325,  respectively,  compared  with  the  control  group  average,  but  it  did 
not  have  a statistically  significant  impact  on  employment  in  those  quarters.  The  fact  that  the 
Riverside  PASS  ERA  program  had  a greater  effect  on  earnings  than  on  employment  suggests 
that  the  program  may  have  led  to  advancement. 

The  Riverside  PASS  ERA  program  produced  gains  in  employment  and  earnings  through 
the  end  of  the  four-year  follow-up  period  (Appendix  Table  C.14).  In  fact,  the  program  had  its 
largest  effect  on  earnings  in  the  fourth  year  of  follow-up,  suggesting  that  the  program  may  lead  to 
even  longer-term  gains. 

In  the  cumulative  four-year  follow-up  period,  the  Riverside  PASS  ERA  program 
produced  gains  in  earnings  and  several  measures  of  advancement  for  the  program  group, 
compared  with  the  control  group.  The  program  increased  average  annual  earnings  by  almost 
$900,  or  by  10  percent  of  control  group  earnings  (Table  5.6).  It  also  increased  the  percentage  of 
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Earnings  ($)  Percentage  employed 


The  Employment  Retention  and  Advancement  Project 


Figure  5.1 

Impacts  on  Employment  and  Earnings  Over  Time: 
Riverside  PASS 


SOURCE:  MDRC  calculations  from  administrative  records  for  the  State  of  California. 

NOTES:  "RA"  refers  to  the  quarter  of  random  assignment. 

Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings. 
Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  and  * = 10 
percent. 
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quarters  in  which  program  group  members  were  earning  more  than  $3,500  by  almost  14  percent 
of  the  control  group  level,  and  it  increased  the  percentage  making  over  $10,000  per  year,  on 
average,  in  Years  1 to  4 by  about  18  percent  of  the  control  group  level  (Appendix  Table  C.15).  In 
addition,  the  nonexperimental  measure  of  average  earnings  per  quarter  employed  is  higher  for 
program  group  members  than  for  control  group  members,  which  provides  another  piece  of 
evidence  that  there  may  have  been  some  advancement  gain  for  program  group  members 
(Appendix  Table  C.15).  The  program  did  not  have  a statistically  significant  impact,  however,  on 
one  of  the  main  measures  of  long-term  advancement:  the  percentage  of  people  who  made  at  least 
$250  or  more  in  their  highest-earning  quarter  in  the  fourth  year  of  follow-up  as  compared  with 
their  highest-earning  quarter  in  the  first  year. 

Table  5.7  shows  that,  despite  large  and  sustained  impacts  on  measures  based  on  UI 
records,  the  Riverside  PASS  ERA  program  did  not  generate  statistically  significant  impacts  for  the 
full  sample  on  most  of  the  broader  measures  of  advancement  examined  in  the  42 -month  survey, 
including  the  benefits,  job  schedules,  job  duties,  or  work  environments  of  sample  members’  most 
recent  jobs.  The  program  did  lead  to  beneficial  impacts  on  the  wage  distribution  in  these  jobs, 
however,  decreasing  the  percentage  of  program  group  members  making  low  hourly  wages  in  the 
job  that  they  held  at  the  time  of  the  42 -month  survey.  In  addition,  the  program  led  to  gains  in 
many  measures  of  advancement  for  a cohort  that  enrolled  later  in  the  study  (discussed  below). 

Public  Assistance  and  Average  Annual  Income 

Driven  by  the  earnings  impacts,  the  Riverside  PASS  ERA  program  increased  average 
annual  income  by  nearly  $800,  or  by  7 percent  of  the  control  group  income  level  (Table  5.6). 
The  earnings  gains  did  not  lead,  however,  to  decreases  in  the  amounts  of  public  assistance 
received  by  program  group  members;  the  program  did  not  affect  TANF  or  food  stamp  benefits 
in  the  four-year  follow-up  period  (Table  5.6). 

Economic  Impact  Findings,  by  Study  Cohort 

Because  much  can  change  over  the  course  of  program  operations  — including  changes 
in  the  program  or  control  services,  in  the  composition  of  the  sample,  and  in  the  external 
environment  — impacts  were  examined  separately  by  study  cohort  for  a number  of  the  ERA 
programs.  This  analysis  was  considered  especially  interesting  for  the  Riverside  PASS  ERA 
program  because  early  implementation  research  suggested  that  program  group  members  may 
not  have  been  receiving  substantially  more  services  than  control  group  members,  and  yet  the 
program  produced  increases  in  several  of  the  key  economic  outcomes  examined  in  this  study.  In 
addition,  budgetary  problems  caused  Riverside  PASS  ERA  program  services  to  be  curtailed  six 
months  earlier  than  planned,  with  the  result  that  some  individuals  enrolling  later  in  the  study 
were  eligible  for  program  services  for  less  than  one  year.  However,  further  analysis  of  auto- 
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The  Employment  Retention  and  Advancement  Project 
Table  5.7 

Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

96.1 

96.3 

-0.1 

0.923 

Currently  employed 

62.5 

62.0 

0.6 

0.860 

No  longer  employed 

33.6 

34.3 

-0.7 

0.826 

Hourlv  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.6 

1.1 

0.5 

0.541 

$5.00 -$6.99 

3.6 

8.8 

0.003 

$7.00  - $8.99 

13.8 

13.9 

-0.1 

0.975 

$9.00  or  more 

43.2 

37.4 

5.7  * 

0.100 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

9.8 

9.9 

0.0 

0.989 

30-34 

6.3 

8.7 

-2.4 

0.190 

35-44 

35.1 

33.9 

1.3 

0.700 

45  or  more 

11.1 

9.5 

1.6 

0.460 

Schedule 

Typical  schedule  (%) 

Regular 

42.9 

39.7 

3.2 

0.350 

Evening  shift 

5.0 

6.2 

-1.1 

0.486 

Other  schedule 

12.8 

14.5 

-1.7 

0.486 

Workweek  includes  at  least  1 weekend  day  (%) 

28.3 

30.4 

-2.1 

0.504 

Benefits 

Employer  offers  (%) 

Sick  days  with  full  pay 

33.0 

30.5 

2.5 

0.437 

Paid  vacation 

39.7 

36.9 

2.8 

0.417 

Paid  holidays  other  than  Christmas  and  New  Year 

36.9 

34.7 

2.3 

0.495 

A health  plan  or  medical  insurance 

38.1 

40.4 

-2.3 

0.495 

None  of  the  above 

16.7 

14.1 

2.6 

0.326 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

51.2 

51.0 

0.2 

0.948 

Receive  respect  from  coworkers 

55.3 

52.1 

3.2 

0.356 

Receive  proper  equipment  needed  to  do  job 

57.6 

57.8 

-0.1 

0.969 

Are  allowed  to  contribute  ideas 

53.1 

54.2 

-1.1 

0.742 

Can  count  on  keeping  job 

22.2 

18.9 

3.3 

0.242 

Think  job  requires  a lot  of  responsibility 

58.8 

58.3 

0.5 

0.889 

Think  job  is  physically  demanding 

30.5 

33.5 

-3.0 

0.366 

Risk  health  or  safety 

23.3 

20.8 

2.5 

0.400 

(continued) 
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Table  5.7  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Are  members  of  labor  union  (%) 

10.8 

12.5 

-1.7 

0.440 

Have  possibility  of  a promotion  (%) 

38.8 

40.4 

-1.6 

0.641 

Duties/reauirements,  at  least  once  Der  month  (%  reporting) 

Reading  and  writing  skills 

45.5 

47.9 

-2.4 

0.476 

Work  with  computers 

32.2 

31.2 

1.1 

0.735 

Arithmetic  skills 

36.1 

32.1 

4.0 

0.217 

Customer  contact 

53.6 

55.7 

-2.1 

0.543 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

33.6 

34.3 

-0.7 

0.826 

0-15 

32.7 

32.7 

0.0 

0.998 

16-30 

20.7 

18.2 

2.4 

0.385 

31-45 

5.2 

6.3 

-1.2 

0.473 

46  or  more 

3.7 

4.5 

-0.8 

0.569 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

48.0 

46.3 

1.6 

0.641 

Ever  received  a promotion  (%) 

28.5 

27.7 

0.8 

0.790 

Found  a different  job  while  working  (%) 

36.5 

32.6 

3.9 

0.245 

Left  a job  to  go  into  a higher-paying  one  (%) 

29.4 

31.7 

-2.4 

0.468 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

38.6 

34.9 

3.7 

0.276 

Earnings 

38.1 

35.4 

2.7 

0.435 

Benefits 

28.7 

27.4 

1.3 

0.676 

Number  of  hours 

34.8 

30.9 

3.9 

0.242 

Start  and  end  of  workday 

38.7 

34.8 

3.9 

0.249 

Commuting  time 

27.9 

26.0 

1.9 

0.552 

Job  security 

33.8 

31.3 

2.4 

0.466 

Opportunity  to  advance 

34.3 

29.7 

4.7 

0.159 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42 -Month  Survey  are  presented  in 
Appendix  D. 
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mated  participation  data  found  that  these  cutbacks  did  not  affect  the  extent  to  which  program 
group  members  received  services. 

The  economic  gains  produced  by  the  Riverside  PASS  ERA  program  were  concentrated 
among  those  who  entered  the  study  in  the  second  half  of  the  enrollment  period  (the  “late 
cohort”)  rather  than  people  who  entered  the  study  in  the  first  half  of  the  enrollment  period  (the 
“early  cohort”).  Program  group  members  in  the  late  cohort  experienced  gains  in  several 
outcomes,  relative  to  the  control  group,  in  the  cumulative  four-year  follow-up  period.17  As  can 
be  seen  in  Table  5.8,  the  program  generated  increases  in  almost  all  measures  of  employment, 
employment  retention,  earnings,  and  advancement  that  were  examined  for  the  late  cohort, 
compared  with  the  levels  for  the  control  group.  For  the  late  cohort,  the  program  increased 
average  annual  earnings  by  $1,500,  or  by  19  percent  of  control  group  earnings.  It  also  increased 
the  percentage  of  quarters  in  which  program  group  members  were  earning  $3,500  or  more  by 
almost  7 percentage  points,  or  by  25  percent  of  the  control  group  level.  The  program  did  not 
produce  any  statistically  significant  impacts  on  any  of  these  measures  for  the  early  cohort. 

It  is  unclear  why  the  impacts  were  larger  for  the  late  cohort.  One  possibility  is  that  the 
composition  of  the  sample  changed.  In  particular,  because  control  group  earnings  were  lower 
for  the  late  cohort  than  for  the  early  cohort,  a couple  of  tests  were  run  to  see  whether  changes  in 
the  composition  of  the  sample  over  time  contributed  to  the  differential  economic  effects  by 
study  cohort.18  Neither  test  suggests  that  this  was  the  case.  Another  possibility  is  that  the  late 
cohort  experienced  larger  participation  impacts.  Unfortunately,  because  12-month  survey 
participation  data  are  available  only  for  the  early  cohort,  it  is  not  known  how  large  participation 
differences  were  for  the  late  cohort.19 

The  42-month  survey  also  provides  evidence  that  the  Riverside  PASS  ERA  program 
was  more  successful  for  the  late  cohort.  As  Table  5.9  shows,  for  several  outcomes  measured  in 
the  42 -month  survey,  program  impacts  for  the  late  cohort  are  statistically  significant,  when  the 
impacts  for  the  full  sample  are  not.  Among  the  late  cohort,  for  example,  the  program  increased 
the  percentage  of  program  group  members  who  were  earning  $9  per  hour  or  more  at  the  job 
they  held  at  the  time  of  the  42-month  survey  by  almost  9 percentage  points,  or  by  26  percent  of 
the  control  group  level.  For  the  late  cohort,  the  program  also  increased  the  percentage  of 


17There  are  statistically  significant  differences  between  the  economic  impacts  for  the  early  and  late  cohorts 
of  sample  members. 

18First,  baseline  characteristics  were  compared  for  sample  members  in  the  early  and  late  cohorts;  no 
differences  were  found.  Second,  a subgroup  analysis  found  that  the  program  produced  gains  for  the  late  cohort 
for  both  the  groups  with  above  and  below-average  earnings  prior  to  study  entry.  For  the  early  cohort,  the 
program  produced  gains  only  for  the  group  with  above-average  earnings  prior  to  study  entry. 

19 As  described  elsewhere  in  the  chapter,  the  automated  participation  data  are  not  used  to  measure  the 
impacts  on  service  receipt  because  they  only  capture  participation  that  took  place  at  a provider’s  location. 
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The  Employment  Retention  and  Advancement  Project 
Table  5.8 


Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4),  by  Cohort: 

Riverside  PASS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Earlv  cohort 

Employment  retention  measures 

Ever  employed  (%) 

89.8 

89.7 

0.1 

0.948 

t 

Average  quarterly  employment  (%) 

59.2 

57.9 

1.4 

0.426 

t 

Had  employment  spell  of  at  least  four  quarters  (%) 

71.2 

71.2 

0.0 

0.995 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

9,789 

9,416 

373 

0.397 

f 

Average  annual  earnings  of  $10,000  or  more  (%) 

42.1 

38.2 

3.9  * 

0.100 

Quarters  with  earnings  of  $3,500  or  more  (%) 

33.6 

31.6 

2.1 

0.209 

t 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

1,241 

1,162 

79 

0.398 

t 

Average  annual  food  stamps  received  ($) 

909 

856 

53 

0.384 

tt 

Average  annual  income0  ($) 

11,939 

11,435 

505 

0.235 

Sample  size  (total  = 1,537) 

899 

638 

Late  cohort 

Employment  retention  measures 

Ever  employed  (%) 

91.6 

87.5 

4. 1 ** 

0.015 

t 

Average  quarterly  employment  (%) 

60.3 

54.1 

6.2  *** 

0.001 

t 

Had  employment  spell  of  at  least  four  quarters  (%) 

72.7 

67.3 

5.4  ** 

0.032 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

9,627 

8,100 

1,526  *** 

0.001 

f 

Average  annual  earnings  of  $10,000  or  more  (%) 

40.1 

31.1 

g 0 

0.001 

Quarters  with  earnings  of  $3,500  or  more  (%) 

33.4 

26.8 

6.6  *** 

0.000 

t 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

1,276 

1,450 

-174 

0.111 

t 

Average  annual  food  stamps  received  ($) 

962 

1,134 

.172  ** 

0.018 

ft 

Average  annual  income0  ($) 

11,864 

10,684 

1,180  *** 

0.008 

Sample  size  (total  = 1,233) 

728 

505 

SOURCE:  MDRC  calculations  from  state  unemployment  insurance  (UI)  wage  records. 

NOTES:  The  early  cohort  was  randomly  assigned  between  July  and  December  2002.  The  late  cohort  was  randomly 
assigned  between  January  and  June  2003. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ttt  = 1 percent;  tt  = 5 percent;  f = 10  percent. 
aThis  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 
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The  Employment  Retention  and  Advancement  Project 
Table  5.9 


Impacts  on  Current  Job  Characteristics  and  Advancement  at  the 
42-Month  Survey  for  the  Early  and  Late  Cohorts: 

Riverside  PASS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Earlv  cohort 

Employment  status 

Ever  employed  since  random  assignment  (%) 

97.7 

96.8 

0.9 

0.643 

Currently  employed 

61.1 

68.0 

-6.8 

0.184 

t 

No  longer  employed 

36.5 

28.8 

7.7 

0.129 

t 

Hourly  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.7 

1.3 

0.5 

0.765 

$5.00 -$6.99 

3.3 

8.3 

-5.1  * 

0.079 

$7.00  - $8.99 

15.1 

16.4 

-1.3 

0.765 

$9.00  or  more 

41.6 

42.6 

-1.0 

0.865 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

10.7 

8.1 

2.7 

0.427 

30-34 

6.5 

10.5 

-4.0 

0.208 

35-44 

35.6 

36.7 

-1.2 

0.819 

45  or  more 

8.1 

12.7 

-4.6 

0.175 

tt 

Sample  size  (total  = 369) 

207 

162 

Late  cohort 

Employment  status 

Ever  employed  since  random  assignment  (%) 

95.0 

95.9 

-0.9 

0.658 

Currently  employed 

62.8 

58.5 

4.4 

0.323 

t 

No  longer  employed 

32.2 

37.4 

-5.2 

0.224 

t 

Hourly  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.5 

1.0 

0.5 

0.616 

$5.00 -$6.99 

3.8 

9.1 

-5.3  ** 

0.020 

$7.00  - $8.99 

13.0 

12.1 

0.9 

0.789 

$9.00  or  more 

43.6 

34.7 

8.9  * 

0.050 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

9.3 

11.0 

-1.7 

0.538 

30-34 

6.3 

7.3 

-0.9 

0.688 

35-44 

34.0 

32.9 

1.0 

0.814 

45  or  more 

13.1 

7.3 

5.8  ** 

0.043 

tt 

Sample  size  (total  = 523) 

297 

226 

(continued) 
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Table  5.9  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  early  cohort  was  randomly  assigned  between  July  and  December  2002.  The  late  cohort  was  randomly 
assigned  between  January  and  June  2003. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statiscal  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  tt  = 5 percent;  | = 10  percent. 


program  group  members  who  were  working  more  than  45  hours  per  week  by  almost  6 percent- 
age points,  or  by  80  percent  of  the  control  group  level.  These  results  provide  evidence  that  at 
least  part  of  the  Ul-based  earning  impacts  produced  by  the  Riverside  PASS  ERA  program  may 
have  been  a result  of  increased  wages  and  hours  worked.  Among  the  late  cohort,  ERA  also 
boosted  the  percentage  of  program  group  members  working  a “regular”  schedule,  the  per- 
centage who  reported  having  an  opportunity  to  advance  at  their  jobs,  and  the  percentage 
regularly  using  computers  and  arithmetic,  compared  with  the  percentages  for  the  control  group 
(Appendix  Table  C.18). 

Economic  Impact  Findings,  by  Service  Provider 

One  of  the  key  features  of  the  Riverside  PASS  ERA  program  was  its  use  of  contractors 
outside  a government  social  service  agency  — three  community-based  organizations  (CBOs) 
and  one  community  college  — to  provide  program  services.  Because  variation  in  service 
provider  was  a key  feature  of  this  program  and  because  random  assignment  of  sample  members 
was  conducted  within  each  provider’s  catchment  area,  impacts  are  presented  separately  by  type 
of  service  provider. 

The  economic  gains  produced  by  the  Riverside  PASS  ERA  program  were  concentrated 
among  people  in  the  areas  served  by  CBOs  rather  than  among  people  in  the  area  served  at  the 
community  college.  (The  sample  that  was  served  by  the  welfare  office  is  not  large  enough  to  be 
included  as  a subgroup.)20  Program  group  members  who  were  served  by  CBOs  experienced 
improvements  in  several  outcomes,  relative  to  their  control  group  counterparts,  in  the 


20There  are  statistically  significant  differences  across  subgroups  based  on  type  of  service  provider. 
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cumulative  four- year  follow-up  period.21  The  CBO-operated  programs  increased  all  measures  of 
employment  retention,  earnings,  and  advancement  that  were  examined  (Appendix  Table  C.19). 
These  programs  increased  average  annual  earnings  by  almost  $1,200,  or  by  14  percent  of 
control  group  earnings.  They  also  increased  the  percentage  of  quarters  in  which  program  group 
members  were  earning  $3,500  or  more  by  6 percentage  points,  or  by  20  percent  of  the  control 
group  level.  The  community  college-operated  program  did  not  generate  any  improvements  in 
the  main  measures  of  employment  retention  or  advancement. 

It  is  not  possible  to  determine  the  specific  reasons  why  the  CBOs  produced  better 
results  than  the  community  college.  One  possibility  is  that  the  CBOs  were  able  to  provide 
stronger  services,  given  their  knowledge  of  their  neighborhoods  and  their  staffs  skill  sets. 
Clients  also  may  have  been  more  willing  to  participate  in  services  at  these  providers  because 
they  viewed  them  as  more  supportive  of  individuals  in  low-income  communities.  Data  that 
could  support  or  refute  these  hypotheses  are  not  available. 

Riverside  PASS  ERA  Program:  Summary  and  Conclusions 

The  Riverside  County  Department  of  Public  Social  Services  (DPSS)  initiated  the  PASS 
ERA  program  in  response  to  the  recognition  that  services  to  help  working  former  TANF 
recipients  stay  employed  and  enhance  their  career  development  were  largely  lacking.  The 
Riverside  PASS  ERA  program  was  a voluntary  program  that  was  designed  to  provide  multifac- 
eted employment  retention  and  advancement  services.  With  a focus  on  individualized  services 
provided  along  with  ongoing  staff-client  relationships,  the  program  offered  job  search  activities, 
career  development  services,  referrals  to  education  and  training  slots,  life  skills  workshops, 
arrangements  for  supportive  service  payments,  and  referrals  to  social  service  programs.  Based 
on  the  assumption  that  private  organizations  were  more  familiar  with  the  jobs  and  services 
available  in  their  communities  than  the  welfare  agency  and  that  individuals  leaving  TANF 
would  be  more  likely  to  work  with  nonwelfare  agencies  other  than  DPSS,  several  community- 
based  organizations  and  a community  college  were  chosen  to  market  and  provide  Riverside 
PASS  ERA  services.  In  contrast,  the  postemployment  services  available  to  control  group 
members  were  not  actively  marketed,  although  control  group  members  could  request  services 
from  designated  welfare  agency  staff  or  could  avail  themselves  of  other  services  offered  in  the 
community. 

By  the  time  Riverside  PASS  ERA  staff  initially  contacted  program  group  members 
following  their  enrollment  in  the  program,  many  had  lost  then  jobs.  Program  staff  actively 


“'The  42 -month  survey  results  do  not  look  different  for  the  sample  served  by  CBOs  than  they  do  for  the 
foil  sample;  there  are  very  few  statistically  significant  differences  between  program  and  control  groups  in  either 
the  hill  sample  or  the  sample  served  by  CBOs  (Appendix  Table  C.20). 
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worked  with  these  individuals  to  reconnect  them  with  employment.  For  individuals  who  were 
working,  staff  provided  individualized  career  development  services,  but  these  services  were 
sometimes  limited  by  the  staffs  skills  and  experience.  Data  from  the  12-month  survey  suggest 
that  Riverside  PASS  ERA  staff  were  unable  to  engage  a substantial  proportion  of  program 
group  members.  Although  program  group  members  participated  in  a job  search  and  education 
and  training  activities  at  fairly  high  rates  within  the  first  year  after  study  entry,  levels  of  receipt 
of  other  types  of  retention  and  advancement  assistance  during  that  time  period  were  much 
lower. 


According  to  the  12-month  survey,  the  Riverside  PASS  ERA  program  generally  did  not 
appear  to  boost  program  group  members’  engagement  or  participation  in  services  above  control 
group  levels.  Program  group  members  did  receive  help  with  career  assessment  at  a rate  of 
almost  triple  the  rate  of  control  group  members,  but  the  absolute  level  of  receipt  of  this  help  was 
relatively  low  for  both  groups. 

Both  participation  levels  and  program  impacts  on  participation,  however,  should  be 
interpreted  with  caution  for  the  Riverside  PASS  ERA  program.  The  participation  data  presented 
above  are  from  the  12-month  survey,  which  was  given  only  to  a subset  of  sample  members 
from  an  early  cohort  of  the  study,  and  few  positive  economic  effects  were  found  for  the  early 
cohort.  Therefore,  the  low  participation  levels  captured  by  the  survey  may  not  reflect  the 
participation  levels  at  the  time  that  the  program  was  producing  economic  benefits  for  program 
group  members.  Analysis  of  data  from  the  program’s  management  information  system  — 
presented  in  the  site-specific  report  on  the  Riverside  PASS  program  (cited  in  Appendix  E)  — 
suggest  that  the  program  may  have  increased  the  use  of  a broader  range  of  services.  These  data 
are  not  used  in  this  report  because  they  only  capture  services  reported  to  the  welfare  department. 

The  Riverside  PASS  ERA  program  increased  several  measures  of  employment 
retention,  earnings,  and  advancement,  relative  to  the  control  group,  over  the  four-year  follow-up 
period.  The  program  produced  gains  in  employment  retention  and  earnings  through  the  end  of 
the  four-year  follow-up  period.  In  fact,  the  program  had  its  largest  effect  on  earnings  in  the 
fourth  year  of  follow-up,  suggesting  that  the  program  may  lead  to  even  longer-term  gains.  The 
Riverside  PASS  ERA  program  did  not  generate  any  impacts  on  public  assistance  receipt  during 
the  follow-up  period,  but  earnings  gains  helped  to  produce  increases  in  average  annual  income. 

The  program’s  effect  on  earnings  primarily  reflects  that  it  helped  the  program  group 
members  get  reemployed  more  quickly  than  the  control  group  members  if  they  left  a job,  rather 
than  extending  the  length  of  time  that  the  program  group  members  were  employed  at  their 
initial  jobs.  There  is  also  evidence  that  the  earnings  gains  for  the  late  cohort  may  have  been  a 
result  of  increased  hours  worked  and  increased  wages.  The  program’s  economic  effects  were 
concentrated  among  sample  members  in  the  areas  served  by  CBOs,  perhaps  reflecting  DPSS’s 
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original  expectation  that  CBOs  had  greater  experience  in  working  with  employed  individuals 
and  better  knowledge  of  the  community  and  the  labor  market  and/or  that  clients  would  be  more 
willing  to  participate  in  services  offered  by  these  organizations. 
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South  Carolina  ERA  Findings 


South  Carolina  ERA  at  a Glance 

• Targeted  employed  and  unemployed  individuals  who  had  left  TANF 

• Provided  individualized  assistance  from  program  staff  for  issues  relating  to  job 
placement,  retention,  and  advancement 

• Was  operated  by  a welfare  agency 

• Was  compared  with  a control  group  that  had  access  to  employment-related  services  in 
the  community,  if  individuals  chose  to  pursue  them 

• Was  implemented  inconsistently  across  county  welfare  offices;  service  delivery  affected 
by  funding  cuts,  management  issues,  and  engagement  challenges 

• Increased  receipt  of  retention  and  advancement  services  relative  to  the  control  group  but 
take-up  rate  very  low 

• Did  not  increase  employment  retention  or  advancement 


South  Carolina  ERA  Test:  Introduction 

The  South  Carolina  ERA  program  sought  to  help  former  welfare  recipients  obtain  jobs, 
increase  employment  retention,  and  move  up  in  the  labor  market.  The  study  enrolled  sample 
members  from  September  2001  through  January  2003,  and  program  services  were  offered 
between  September  2001  and  April  2005  in  six  rural  and  economically  disadvantaged  counties 
in  the  northeast  part  of  the  state. 

Origins  of  the  Test 

The  South  Carolina  ERA  program  (referred  to  as  “Moving  Up”)  was  developed  by  the 
state’s  Department  of  Social  Services  (DSS)  in  response  to  concerns  about  sustained  poverty 
among  those  who  had  left  the  welfare  rolls  — specifically,  that  some  were  not  working;  many 
were  working  but  not  steadily;  and  others  were  in  low-wage  jobs.  Moreover,  prior  to  ERA, 
there  were  no  formal  programs  in  South  Carolina  for  those  who  left  TANF.  Instead,  motivated 
individuals  had  to  seek  out  various  employment-related  services  offered  in  the  community,  as 
they  also  had  to  do  to  access  public  benefits. 

The  South  Carolina  ERA  program  sought  to  enhance  the  services  available  for  those 
who  had  previously  received  TANF  and  to  encourage  individuals  to  utilize  these  services  and 
succeed  in  the  labor  market  through  the  provision  of  modest  financial  incentives.  The  South 
Carolina  ERA  program  offered  ongoing  staff-client  relationships,  with  staff  providing  or 
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connecting  participants  with  a range  of  services,  including  job  search  assistance,  short-term 
vocational  training,  and  support  services. 

Labor  Market  Context 

The  South  Carolina  ERA  program  and  sample  members  faced  challenging  conditions 
for  securing  and  stabilizing  employment  and  advancing  to  better  jobs.  In  South  Carolina, 
random  assignment  began  roughly  in  the  middle  of  the  2001  recession.  At  this  time,  the  area 
had  an  unemployment  rate  of  5.2  percent,  compared  with  the  national  rate  of  4.0  percent.  While 
the  national  rate  peaked  in  2003  at  6.0  percent,  the  rate  in  the  six-county  area  continued  to  grow 
until  2005,  reaching  a high  of  9.6  percent.  The  unemployment  rates  in  counties  included  in  the 
study  began  to  decline  in  2006,  but  the  2007  area  rate  of  7.8  percent  was  higher  than  the  2007 
national  rate  of  4.6  percent. 

Target  Population 

The  South  Carolina  ERA  program  targeted  a broad  group  of  individuals  who  had  left 
TANF.  The  sample  included  all  those  living  in  the  six-county  area  who  had  left  TANF  for  any 
reason  between  October  1997  and  December  2000  and  who  did  not  return  to  the  TANF  rolls. 
By  targeting  people  who  left  TANF  and  had  not  returned  for  a considerable  period,  regardless 
of  whether  they  were  currently  employed  or  not,  the  South  Carolina  ERA  program  chose  a 
target  population  that,  for  the  most  part,  was  making  do  without  services  from  the  TANF 
program. 

Table  5.2  shows  selected  characteristics  of  the  South  Carolina  ERA  sample  members  at 
the  time  the  study  began.  At  random  assignment,  sample  members  had  been  off  TANF  for 
between  nine  months  and  five  years.  Only  a little  more  than  half  the  sample  (56  percent)  had  a 
high  school  diploma  or  a GED  certificate  on  entering  the  study  — the  lowest  rate  of  all  the 
samples  in  the  ERA  tests  discussed  in  this  chapter. 

Significance  of  This  Test 

The  South  Carolina  ERA  test  examines  whether  a voluntary  program  that  is 
administered  through  a welfare  agency  — using  ongoing  staff-client  relationships  to  connect 
participants  with  a range  of  services,  including  job  search  assistance,  short-term  vocational 
training,  support  services,  and  modest  financial  incentives  — can  improve  job  placement, 
retention,  and  advancement  for  individuals  who  had  previously  received  TANF,  compared  with 
the  standard  postemployment  services  available  in  the  community.  Chart  5.5  presents  the 
important  differences  between  the  services  available  to  program  group  and  control  group 
members  in  the  South  Carolina  ERA  test. 
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The  Employment  Retention  and  Advancement  Project 
Chart  5.5 

Program-Control  Group  Differences: 

South  Carolina  ERA  Test 


ERA  Group 

Control  Group 

Goals 

• Place  fonner  welfare  recipients  in  jobs,  help  them 
sustain  employment  and  advance,  and  increase  their 
earnings 

Program 
resources  and 
structures 

• Operated  by  welfare  agency 

• No  single  retention-  and 
advancement-oriented  program  for 
employed  individuals  in  community; 
however,  could  access  various 
employment-related  community 
services  should  they  choose  to  pursue 

Staffing 

• Staff  worked  one-on-one  with  individuals; 
agencywide  workforce  consultants  available 

• Staff  schedules  and  meeting  locations  flexible  for 
clients 

Staff-client 

engagement 

• Aggressive  marketing  and  outreach  (with  modest 
participation  incentives) 

• Services  tailored  to  participants 

• Developed  and  maintained  employment  plans. 

• Limited  ongoing  contacts 

Initial  job 
preparation 
and  placement 

• Multifaceted  help;  preparing  a resume,  one-on-one 
assistance  with  job  search,  referrals  to  job  search 
classes,  and  an  assessment  of  potential  barriers  to 
employment 

• Could  pursue  services  on  their  own; 
however,  similar  reported  rates  of 
participation  in  job  search  activities  as 
ERA  group 

Retention 

services 

• Job  search  assistance  (preparing  resumes  and  job 
applications,  identifying  job  leads) 

• Assistance  with  supportive  services  and  transitional 
benefits;  social  service  referrals  as  needed 

• Discussed  problems  on  the  job 

• Could  pursue  services  on  their  own; 
however,  similar  reported  rates  of 
participation  in  job  search  activities  as 
ERA  group 

• Could  pursue  services  on  their  own; 
however,  similar  reported  rates  of 
receiving  help  with  basic  needs  as 
ERA  group 

Advancement 

services 

• Counseling  for  work-based  advancement 

• Limited  services  for  advancement  through  job 
change 

• Education  and  training  referrals 

SOURCES:  Site-specific  reports.  For  citations,  see  Appendix  E. 


NOTE:  aEmployment-related  and  supportive  services  were  available  through  Workforce  Investment  Act 
providers,  One-Stop  Centers,  community  colleges,  adult  schools  and  other  education  providers,  and 
employment  and  training  organizations. 
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South  Carolina  ERA  Program:  Features  and  Client  Experiences 

Program  Resources  and  Organization 

DSS  developed  and  operated  the  South  Carolina  ERA  program  in  six  counties: 
Chesterfield,  Darlington,  Dillon,  Florence,  Marion,  and  Marlboro.  Administration  of  the 
program  also  included  some  informal  coordination  with  WIA  One-Stop  Centers,  although  this 
varied  over  time  and  across  DSS  offices.  Financed  with  TANF  funds,  the  program  was  fully 
funded  when  the  study  began.  However,  South  Carolina’s  budget  situation  deteriorated  over 
time,  leading  to  funding  cuts.  ERA  staff  remained  on  the  job,  but,  for  a period  from  late  2002 
through  summer  2003,  most  counties  froze  or  limited  spending  for  some  ERA  services, 
including  financial  incentives,  education  and  training  tuition  payments,  and  transportation 
assistance. 

Staffing 

Services  in  the  South  Carolina  ERA  program  were  primarily  provided  by  DSS  staff. 
These  staff  were  responsible  for  developing  ongoing  (one-on-one)  staff-client  relationships  and 
for  connecting  program  group  members  with  other  services  as  needed.  The  largest  county  in  the 
study  had  four  ERA  staff;  the  next-largest  county  had  two;  and  the  other  four  counties  each  had 
one  staff  person.  DSS  workforce  staff  — who  built  relationships  with  local  employers, 
developed  jobs,  and  assembled  job  listings  — were  also  available  to  work  with  the  ERA  staff. 

The  overall  management  of  the  program  was  the  responsibility  of  a full-time  program 
coordinator,  and  DSS  directors  in  each  participating  office  oversaw  the  program’s  daily 
operations.  There  were,  however,  operational  differences  across  the  counties. 

Most  of  the  ERA  staff  had  previously  worked  for  DSS  in  some  capacity  — many,  in  the 
state’s  TANF  program  — and  all  had  some  prior  experience  in  social  services.  But  ERA  staff 
members  and  the  DSS  as  a whole  had  more  experience  working  with  unemployed  individuals, 
rather  than  those  who  were  employed.  As  a result,  new  management  tools  were  developed  for 
the  program.  Performance  standards  were  established  for  ERA  staff  that  focused  on  the  number 
of  contacts  between  staff  and  participants,  the  number  of  individuals  participating  in  the 
program,  the  number  of  individuals  who  were  placed  in  jobs,  and  the  number  who  received  a 
raise  or  increased  their  work  hours.  The  ERA  program  coordinator  also  encouraged  staff  to 
contact  at  least  75  percent  of  their  cases  at  least  once  and  to  keep  at  least  35  percent 
participating  in  the  program  at  any  given  time.  An  automated  client  tracking  system  was 
developed  which  allowed  the  program  coordinator  to  monitor  the  performance  of  staff  and  to 
provide  feedback  to  them  (although  there  were  no  specific  incentives  for  achieving  performance 
goals). 
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The  ERA  staff  were  often  available  — either  in  the  office  or  by  phone  — beyond  the 
standard  9-to-5  workday,  in  order  to  help  program  group  members  who  worked  or  had  other 
responsibilities  during  the  day. 

Staff-Client  Engagement 

Ongoing  staff-client  relationships  were  at  the  core  of  the  South  Carolina  ERA  program. 
Because  these  sample  members  were  not  receiving  TANF,  participation  in  the  ERA  program 
was  entirely  voluntary.  Reflecting  this,  South  Carolina  ERA  management  emphasized  the 
importance  of  effective  outreach  and  marketing,  and  all  ERA  staff  received  training  on 
engaging  clients  in  program  activities. 

Locating  individuals  often  took  considerable  effort,  because  contact  information  in  the 
state’s  database  was  often  outdated.  Once  this  information  was  obtained,  ERA  staff  sent  an 
invitation  letter  that  included  a brochure  about  the  program  and  typically  was  followed  by 
phone  calls  and  additional  letters.  The  South  Carolina  ERA  program  also  used  modest  financial 
incentives  to  encourage  program  participation.  For  example,  individuals  received  $10  for 
attending  an  initial  meeting  with  ERA  staff  and  $50  for  completing  a job  search  class.  Table 
5.10  shows  that,  according  to  the  study’s  12-month  survey,  less  than  half  the  program  group 
members  (43  percent)  reported  having  had  contact  with  employment  program  staff — a rate  that 
is  14  percentage  points  greater  than  the  rate  of  contact  with  employment  program  staff  that  was 
reported  by  control  group  members. 

ERA  staff  faced  challenges  in  marketing  the  program  to  individuals  who  were  not 
required  to  take  part  in  it  and  who  may  not  have  wanted  the  program’s  assistance.  Staff 
encouraged  participation  in  the  South  Carolina  ERA  program  by  describing  its  individualized 
nature  and  emphasizing  clients’  broader  goals  for  themselves  and  their  families.  In  field 
research  interviews,  ERA  staff  reported  that  some  of  the  people  they  spoke  with  said  that  they 
were  “doing  fine”  and  did  not  need  or  want  help. 

After  program  group  members  agreed  to  participate,  ERA  staff  held  meetings  to  assess 
individuals’  employment,  education,  and  family  situations;  discuss  employment  goals  and 
barriers;  and  develop  a Career  Enhancement  Plan.  The  goal  was  to  develop  such  a plan  within 
10  days  of  individuals’  agreement  to  participate.  The  program  did  not  use  standardized 
assessments  of  skills  or  testing,  and  it  relied  on  staff  expertise  to  develop  these  plans,  based  on 
discussions  with  the  clients.  About  8 percent  of  program  group  members  received  a career 
assessment  — roughly  twice  the  rate  reported  by  the  control  group  (Appendix  Table  C.2 1 ) . 

Each  program  group  member’s  activities  and  services  were  based  on  his  or  her  career 
plan.  ERA  staff  worked  with  individuals  to  help  them  prepare  for  and  find  a job,  stay  in  their 
current  job,  or  move  up  to  a better  job.  Program  group  members  were  typically  referred  to 
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The  Employment  Retention  and  Advancement  Project 
Table  5.10 

Summary  of  Impacts: 

South  Carolina 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Implementation  outcomes 

Engagement 

Ever  had  contact  with  staff/employment  program  (%) 

43.0 

29.1 

q *** 

0.001 

Average  number  of  contacts  with  staff 

4.4 

2.7 

1.7  * 

0.077 

Talked  with  staff  in  past  4 weeks  (%) 

16.0 

10.8 

5.3  * 

0.077 

Retention  services 

Participated  in  a job  search  activity  (%) 

42.4 

44.2 

-1.9 

0.654 

Received  help  dealing  with  problems  on  the  job  (%) 

4.6 

2.3 

2.3 

0.153 

Advancement  services 

Received  help  with  career  assessment  (%) 

7.5 

4.1 

3.4  * 

0.094 

Received  help  finding  a better  job  while  working  (%) 

12.8 

3.9 

g q *** 

0.000 

Participated  in  an  education/training  activity3  (%) 

26.0 

19.7 

6.3  * 

0.080 

Received  any  help  with  retention/advancement"  (%) 

17.6 

8.2 

9 q *** 

0.001 

Sample  size  (total  = 552) 

276 

276 

Cumulative  economic  outcomes  f Years  1-4) 

Employment  measures 

Ever  employed  (%) 

74.9 

75.1 

-0.2 

0.875 

Average  quarterly  employment  (%) 

48.1 

47.0 

1.1 

0.303 

Had  employment  spell  of  at  least  4 quarters  (%) 

55.6 

55.6 

0.1 

0.974 

Length  of  longest  employment  spell,  in  quarters 

6.5 

6.3 

0.2 

0.281 

Length  of  longest  unemployment  spell,  in  quarters 

7.3 

7.5 

-0.2 

0.210 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

6,054 

6,121 

-67 

0.733 

Quarters  with  earnings  of  $3,500  or  more  (%) 

18.1 

18.5 

-0.3 

0.686 

Comparison  of  quarter  with  highest  earnings  in  Year  1 
and  quarter  with  highest  earnings  in  Year  4 (%) 

Not  employed  in  one  or  both  years 

50.7 

51.2 

-0.5 

0.737 

Earnings  decreased  by  more  than  $250 

16.2 

16.0 

0.2 

0.879 

Earnings  changed  by  less  than  $250 

7.3 

6.4 

0.9 

0.347 

Earnings  increased  by  $250  or  more 

25.9 

26.3 

-0.4 

0.780 

Public  assistance  and  income0 

Average  annual  TANF  received  ($) 

74 

67 

8 

0.403 

Average  annual  food  stamps  received  ($) 

1,973 

1,982 

-9 

0.866 

Average  annual  income3  ($) 

8,157 

8,217 

-60 

0.760 

Sample  size  (total  = 2,776) 

1,382 

1,394 

(continued) 
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Table  5.10  (continued) 

SOURCES:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey  and  state  administrative 
records. 

NOTES:  Averages  for  dollar  amounts  include  zero  values  for  sample  members  who  had  no  earnings,  TANF 
grants,  or  food  stamp  benefits. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes,  college 
courses,  or  vocational  training. 

bThis  measure  includes  numerous  forms  of  retention  and  advancement  help,  including  help  finding  a better 
job  while  working,  enrolling  in  life  skills  classes,  help  with  career  assessment,  help  with  problems  on  the  job, 
and  addressing  personal  problems. 

cData  on  public  assistance  are  missing  for  156  sample  members.  Therefore,  the  measure  for  public 
assistance  and  income  is  available  only  for  2,620  sample  members. 

dThis  measure  represents  the  sum  of  Ul-reported  earnings,  TANF,  and  food  stamps. 


structured  activities,  such  as  job  search  classes  and  short-term  vocational  training.  ERA  staff 
were  expected  to  meet  with  participants  at  least  once  a month  to  monitor  their  progress. 

Initial  Job  Preparation  and  Placement  Services 

Field  research  suggests  that  job  placement  was  the  strongest  component  of  the  South 
Carolina  ERA  program.  This  was  due,  in  part,  to  the  considerable  experience  that  DSS  and  most 
staff  members  had  with  helping  people  prepare  for  and  find  jobs.  Typically,  program  group 
members  who  were  not  working  received  assistance  preparing  for  and  searching  for  a job,  such 
as  help  preparing  a resume,  one-on-one  assistance  with  the  job  search  process,  referrals  to  job 
search  classes  at  DSS,  and  an  assessment  of  potential  barriers  to  employment.  However,  while 
more  than  40  percent  of  the  program  group  members  reported  in  the  12-month  survey  that  they 
had  participated  in  a job  search  activity,  this  rate  is  no  different  from  the  rate  reported  by  control 
group  members. 

Retention  Sen/ices 

The  South  Carolina  ERA  program  included  multiple  tools  for  stabilizing  employment 
among  program  group  members,  including  financial  incentives,  assistance  with  work  supports, 
referrals  to  social  services,  and  coaching  on  the  resolution  of  work-place  problems  (see  below). 
As  discussed,  South  Carolina  ERA  staff  had  little  experience  serving  employed  individuals. 
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Despite  training  in  this  area,  delivery  of  retention  services  remained  challenging  and  was 
inconsistent  across  the  six  counties. 

The  ERA  program  in  South  Carolina  provided  modest  financial  incentives  to  encourage 
and  reward  employment-related  achievements,  including  holding  a job  and  getting  a promotion. 
The  program  established  benchmarks  of  providing  $50  to  participants  if  they  remained  in  a new 
job  for  a month,  another  $50  after  three  months,  and  another  $50  after  six  months.  Likewise,  the 
program  provided  financial  incentives  for  advancing  in  the  labor  market,  including  $50  to 
participants  who  increased  their  wage  by  8 percent  or  more,  moved  from  a part-time  job  to  a 
full-time  one,  or  moved  to  a job  with  benefits.  While  these  incentive  payments  were  made 
during  the  early  parts  of  the  study  period,  the  state’s  budget  problems  prevented  most  counties 
from  paying  any  incentives  between  late  2002  and  mid-2003.  Overall,  47  percent  of  the 
program  group  received  at  least  one  incentive  payment  within  a year  after  entering  the  study, 
and  16  percent  received  at  least  one  payment  of  $50  or  more.  Among  individuals  who  received 
at  least  one  payment,  the  average  amount  received  during  the  year  was  $62. 

Assistance  with  supportive  services  and  transitional  benefits  was  also  provided  in  the 
South  Carolina  ERA  program.  As  public  transportation  in  the  study  region  was  limited, 
transportation  assistance  was  an  important  component  of  the  program.  South  Carolina  ERA 
staff  also  referred  program  group  members  to  mental  health  counseling,  substance  abuse 
treatment,  or  services  for  victims  of  domestic  violence,  and  some  participants  obtained  TANF, 
food  stamps,  Medicaid,  and  other  work  supports.  While  it  was  intended  that  the  program  would 
have  funds  available  for  child  care,  state  budget  problems  prevented  the  allocation  of  those 
funds.  The  12-month  client  survey  found  that  the  ERA  program  resulted  in  an  increase  in  the 
receipt  of  support  services:  22  percent  of  the  program  group  reported  receiving  such  help  — a 
rate  that  is  8 percentage  points  greater  than  the  rate  reported  by  the  control  group  (Appendix 
Table  C.21). 

To  foster  job  retention,  ERA  staff  coached  program  group  members  on  workplace 
issues  and  held  periodic  check-ins  to  allow  participants  to  discuss  work-related  concerns.  Based 
on  the  12-month  survey,  however,  less  than  5 percent  of  program  group  members  reported 
receiving  help  with  problems  on  the  job  during  their  first  year  in  the  program  (Table  5.10). 

Advancement  Sen/ices 

Advancement  to  better  jobs  was  one  of  the  goals  of  the  South  Carolina  ERA  program. 
Although  the  delivery  of  these  services  was  difficult  and  inconsistent  across  the  six  program 
offices,  it  could  include  instruction  in  how  to  approach  a supervisor  to  discuss  a raise  or  how  to 
learn  about  promotion  opportunities.  In  field  research  interviews,  ERA  staff  reported  that  it  was 
challenging  to  convince  working  individuals  to  participate  in  postemployment  services  — 
particularly  advancement  services  — because  they  were  juggling  work,  family,  and  other 


226 


responsibilities  or  they  were  comfortable  in  their  current  job  and  did  not  want  to  switch  jobs. 
The  12-month  survey  shows  that  the  program  produced  gains  in  the  take-up  of  retention  and 
advancement  services;  1 8 percent  of  program  group  members  reported  that  they  received  help 
with  retention  and  advancement  — more  than  double  the  rate  reported  by  the  control  group 
(Table  5.10). 

The  South  Carolina  ERA  program  included  some  referrals  for  both  employed  and 
unemployed  program  group  members  to  short-term  education  and  training.  For  example,  if  a 
program  group  member  did  not  have  a high  school  diploma  or  a GED  certificate,  staff  would 
recommend  GED  preparation  classes,  if  appropriate.  The  12-month  survey  shows  that  the 
program  produced  a small  increase  in  participation  in  education  and  training. 

Employer  Linkages 

There  was  no  formal  connection  between  the  South  Carolina  ERA  program  and  local 
employers.  ERA  staff  did  not  have  close  connections  with  employers  and  did  not  work  to 
identify  local  job  opportunities  that  would  be  appropriate  for  individuals  on  their  caseload.  ERA 
staff  did  work  with  DSS  job  development  staff  located  in  each  of  the  welfare  offices.  These 
DSS  staff  identified  job  openings  at  local  employers,  primarily  for  the  benefit  of  TANF  clients, 
although  they  often  shared  job  openings  with  the  ERA  staff. 

South  Carolina  ERA  Program:  Economic  Impacts 

Economic  impacts  of  the  South  Carolina  ERA  program  can  be  summarized  as  follows: 

• The  South  Carolina  ERA  program  did  not  generate  any  changes,  relative  to 
the  control  group,  in  the  main  measures  of  employment  or  employment 
retention  for  the  full  sample  in  the  four-year  follow-up  period.  In  addition, 
there  is  no  evidence  that  the  program  led  to  earnings  or  advancement  gains, 
compared  with  the  levels  seen  in  the  control  group. 

• The  South  Carolina  ERA  program  did  not  produce  any  changes  in  the 
amount  of  TANF  or  food  stamp  assistance  received  or  in  participants’ 
average  annual  income. 

Control  Group  Outcomes 

Average  quarterly  employment  and  average  earnings  for  control  group  members  in  the 
South  Carolina  ERA  test  were  much  lower  than  the  levels  found  in  the  other  control  groups 
examined  in  Chapters  4 and  5,  reflecting  that  many  were  not  employed  at  the  time  they  entered 
the  study.  Average  quarterly  employment  was  less  than  50  percent,  and  average  earnings  were 
just  over  $6,000  per  year  for  control  group  members  over  the  four-year  follow-up  period  (Table 
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5.10).  Average  annual  earnings  and  average  quarterly  employment  remained  fairly  steady  over 
the  follow-up  period  (Appendix  Table  C.22).  Less  than  10  percent  of  the  control  group  received 
at  least  one  TANF  payment  in  each  of  the  three  years  of  follow-up  (Appendix  Table  C.22). 

Employment  and  Employment  Retention 

The  South  Carolina  ERA  program  did  not  have  any  effects,  relative  to  the  control 
group,  on  the  main  measures  of  employment  or  employment  retention  in  the  four-year  follow- 
up period  (Table  5.10). 

Earnings  and  Advancement 

The  South  Carolina  ERA  program  did  not  lead  to  increases  in  any  measures  of  earnings 
or  advancement,  compared  with  the  control  group  (Table  5.10). 

Public  Assistance  and  Average  Annual  Income 

The  South  Carolina  ERA  program  had  no  effect,  relative  to  the  control  group,  on  the 
receipt  of  TANF  or  food  stamp  assistance  or  participants’  average  annual  income  during  the 
cumulative  four-year  follow-up  period  (Table  5.10). 

Economic  Impact  Findings,  by  Prior  Labor  Force  Attachment 

The  South  Carolina  ERA  program  was  the  only  program  in  the  ERA  study  that 
explicitly  targeted  both  employed  and  unemployed  individuals.  Therefore,  impacts  were 
examined  for  sample  members  who  had  a lot,  some,  or  no  employment  in  the  year  prior  to  study 
entry.  This  analysis  shows  that  there  were  statistically  significant  increases,  relative  to  the 
control  group,  in  average  quarterly  employment  and  average  earnings  for  program  group 
members  who  had  some  recent  labor  market  experience  (employed  one  to  three  quarters  in  the 
year  prior  to  study  entry)  (Appendix  Table  C.25).  However,  program  group  members  who  were 
employed  in  either  none  or  all  of  the  quarters  in  the  year  prior  to  study  entry  experienced  no 
gains,  compared  with  their  counterparts  in  the  control  group. 

Economic  Impact  Findings,  by  Study  Cohort 

Despite  the  funding  problems  that  occurred  midway  through  operations  in  the  South 
Carolina  ERA  program,  there  are  no  statistically  significant  differences  in  impacts  on  the  main 
measures  of  employment,  employment  retention,  earnings,  or  advancement  when  analyzed  by 
study  cohort  (not  shown). 
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South  Carolina  ERA  Program:  Summary  and  Conclusions 

The  South  Carolina  ERA  program  was  developed  by  the  state’s  Department  of  Social 
Services  (DSS)  in  response  to  concerns  about  sustained  poverty  among  those  who  had  left 
TANF  and  about  the  lack  of  programs  and  services  available  to  this  population.  The  program 
was  voluntary  and  was  operated  by  welfare  agency  staff.  Building  on  ongoing  staff-client 
relationships,  the  model  included  connecting  program  group  members  with  a range  of  services. 
Although  no  formal  services  were  provided  to  the  control  group,  these  sample  members  could 
access  standard  services  available  in  the  community. 

Ongoing  staff-client  relationships  represented  the  core  of  the  South  Carolina  ERA 
program.  Staff  worked  with  participants  to  understand  their  employment  goals  and  develop  an 
employment  plan.  Staff  also  provided  or  connected  participants  with  a range  of  services 
(including  one-on-one  job  search  assistance,  job  search  classes,  and  short-term  vocational 
training)  and  support  services  (such  as  transportation  assistance).  However,  the  program’s 
services  varied  somewhat  across  the  six  counties  that  operated  it.  Funding  cutbacks  partway 
through  the  program  as  well  as  limits  to  staff  training  and  experience  resulted  in  limits  on  some 
services. 

Despite  extensive  outreach  efforts,  less  than  half  the  program  group  members  reported 
that  they  had  had  contact  with  employment  program  staff  within  their  first  year  after  study 
entry.  Maintaining  ongoing  engagement  proved  even  more  challenging,  as  less  than  one- fifth  of 
program  group  members  reported  having  contact  with  ERA  staff  or  staff  from  an  employment 
program  within  four  weeks  prior  to  the  12-month  survey.  While  over  a third  of  program  group 
members  participated  in  a job  search  activity,  less  than  one- fifth  of  them  reported  receiving  help 
with  retention  and  advancement  in  the  first  year. 

Even  though  levels  of  engagement  among  members  of  the  program  group  were  fairly 
low,  the  rates  of  contact  and  recent  contact  with  ERA  staff  or  staff  from  an  employment 
program  were  greater  than  those  reported  by  control  group  members.  In  addition,  the  program 
increased  the  proportion  of  program  group  members  who  reported  receiving  help  with  retention 
and  advancement  — to  more  than  double  the  rate  reported  by  the  control  group,  although  the 
overall  levels  remained  low  for  both  groups.  Although  job  search  assistance  was  an  important 
aspect  of  the  South  Carolina  ERA  program,  the  program  failed  to  increase  rates  of  participation 
in  this  activity. 

In  the  four-year  follow-up  period,  the  South  Carolina  ERA  program  did  not  generate 
any  changes,  relative  to  the  control  group,  in  employment,  employment  retention,  earnings,  or 
advancement. 

The  South  Carolina  ERA  program  encountered  several  implementation  challenges.  It 
was  difficult  for  staff  to  address  the  needs  of  both  employed  and  unemployed  individuals  who 
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had  left  TANF.  Staff  did  not  appear  to  have  the  appropriate  skills  to  address  the  different  needs 
of  the  relatively  broad  target  group,  despite  the  training  that  was  provided.  In  particular, 
providing  services  to  those  who  were  working  was  new  to  most  staff  involved  in  the  program. 
Funding  constraints  made  it  difficult  to  offer  the  full  range  of  services,  particularly  financial 
incentives  for  participation.  And  some  potential  participants  were  hard  to  locate  and  physically 
dispersed,  making  outreach  and  recruitment  efforts  difficult.  Furthermore,  the  poor  state 
economy  and  the  focus  on  rural  areas  with  limited  job  opportunities  may  have  made  it  difficult 
for  the  program  to  help  people  find  jobs,  or  find  better  jobs  than  the  ones  that  they  held  at  the 
time  of  the  study.  Finally,  it  may  be  that  the  individuals  who  were  targeted  for  this  intervention 
— people  who  had  left  TANF  up  to  several  years  earlier  — were  in  less  immediate  need  of 
services  than  other  potential  target  populations  might  have  been. 
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Chapter  6 

Summary  and  Conclusions 


This  chapter  summarizes  the  findings  for  the  Employment  Retention  and  Advancement 
(ERA)  programs,  grouping  them  by  target  populations,  and  then  presents  a cross-site  summary 
and  the  conclusions  of  the  report. 


Summary  of  Findings,  by  Target  Group 

This  section  presents  a summary  of  the  findings  for  each  of  the  three  groupings  of  ERA 
programs  discussed  in  Chapters  3 through  5.  First,  the  programs’  overall  service  strategies  are 
discussed.  Then,  impacts  on  contact  and  engagement  with  clients  are  summarized.  This  is 
followed  by  a review  of  impacts  on  “traditional”  employment  and  education  services  (defined  in 
this  study  as  job  search  and  education  and  training).  The  patterns  of  findings  for  these  services 
are  different  from  the  patterns  for  other  retention  and  advancement  services,  so  they  are  de- 
scribed separately.  Finally,  each  of  the  target  group  summaries  concludes  with  a discussion  of 
economic  impacts. 

Programs  Serving  Unemployed  TANF  Recipients 

As  discussed  in  Chapter  3,  the  Texas,  Los  Angeles  Enhanced  Job  Club  (EJC),  and  Sa- 
lem ERA  programs  served  unemployed  TANF  recipients. 

Service  Strategies 

The  programs  serving  this  target  group  had  some  common  strategies,  primarily  provid- 
ing preemployment  services  (such  as  job  preparation  and  job  search)  in  which  individuals  were 
mandated  to  participate  as  a condition  of  receiving  cash  assistance.  However,  the  programs  also 
had  notable  variation.  For  example,  the  Los  Angeles  EJC  model  did  not  include  any  postem- 
ployment services,  by  design,  but  hoped  to  achieve  retention  and  advancement  goals  by  initially 
placing  participants  in  jobs  in  their  career  of  interest.  In  contrast,  the  Salem  and  Texas  ERA 
programs  included  postemployment  services  through  ongoing  staff-client  interactions  as  a key 
component.  The  Texas  program  was  unique  in  this  target  group  (and  in  ERA)  in  that  it  provided 
a substantial  financial  incentive  of  $200  per  month  to  those  who  left  Temporary  Assistance  for 
Needy  Families  (TANF  leavers)  and  were  working  full  time. 
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Contact  and  Engagement 

As  shown  in  the  set  of  bars  to  the  left  in  Figure  6.1,  among  the  programs  serving  unem- 
ployed TANF  recipients,  only  the  Corpus  Christi  and  Fort  Worth  sites  operating  the  Texas  ERA 
program  produced  increases  in  contacts  with  staff  of  a welfare  or  employment  program.  These 
sites  also  increased  ongoing  contact  (as  measured  by  the  percentage  of  sample  members  who 
spoke  with  staff  in  the  four  weeks  prior  to  responding  to  the  12-month  survey).  While  the  Salem 
program  did  not  increase  the  percentage  who  ever  had  a contact,  it  did  increase  the  overall 
number  of  contacts  (not  shown). 1 The  Houston  and  Los  Angeles  EJC  ERA  programs  did  not 
increase  any  measures  of  client  engagement. 

Traditional  Employment  and  Education  Services 

The  set  of  bars  to  the  left  in  Figure  6.2  shows  that  the  rates  of  participation  in  job  search 
by  control  group  members  in  these  three  ERA  programs  were  quite  high,  which  is  not  surpris- 
ing, because  these  programs  served  unemployed  people.  The  program  in  Houston  was  the  only 
one  to  produce  increases  in  job  search  participation  above  the  control  group  levels.  Control 
group  levels  of  participation  in  education  and  training  were  also  relatively  high,  and  no  program 
groups  exceeded  these  levels. 

Other  Retention  and  Advancement  Sen/ices 

Two  measures  of  postemployment  retention  and  advancement  “help”  are  shown  in  Fig- 
ure 6.3:  the  percentage  of  people  reporting  that  they  received  help  finding  a better  job  while 
working  and  the  percentage  of  people  reporting  that  they  received  help  with  problems  on  the 
job.  Take-up  of  these  services  was  very  low  (close  to  zero)  among  the  control  groups.  For  these 
measures  of  retention  and  advancement  help,  participation  rates  among  ERA  group  members 
generally  ranged  from  the  single  digits  to  under  20  percent.  Given  that  the  focus  of  the  ERA 
programs  was  on  providing  assistance  with  retention  and  advancement,  the  rates  of  receipt  of 
these  types  of  assistance  among  ERA  program  group  members  were  fairly  low.2  The  Corpus 
Christi.  Fort  Worth,  and  Salem  sites  produced  small-to-moderate  impacts  on  the  receipt  of  these 
services,  while  the  Houston  site  did  not.  Take-up  of  the  Texas  stipend  varied  by  site:  30  percent 
of  all  program  group  members  took  up  the  stipend  in  Corpus  Christi,  compared  with  20  percent 
in  the  other  two  Texas  sites. 


'Molina,  Cheng,  and  Hendra  (2008). 

2Figure  6.3  and  most  of  this  section  focus  on  only  two  measures  of  postemployment  retention  and  ad- 
vancement services;  results  for  other  such  measures  that  are  presented  in  Chapters  3,  4,  and  5 show  similar 
patterns. 
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The  Employment  Retention  and  Advancement  Project 
Figure  6.1 

Impacts  on  Client  Engagement 


Ever  had  contact  with  staff/employment  program  (%) 

□ Impacts  ■ Control  group  levels 


Unemployed  TANF  recipients  Employed  TANF  recipients  Employed  non-TANF  recipients 

(continued) 
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Figure  6.1  (continued) 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  See  site  summary  tables  for  the  supporting  numbers  for  these  figures. 

A two-tailed  t-test  was  applied  to  differences  between  outcomes  for  the  program  and  control  groups. 
Statistical  significance  levels  are  indicated  as:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  are  all  part  of  the  Texas  ERA  model. 

The  outcome  measures  are  discussed  in  Chapter  2. 

The  survey  samples  are  small  in  Fort  Worth  and  Riverside  PASS,  and  some  response  bias  was  noted 
in  Riverside  PASS  and  Medford.  See  Appendix  D. 


Economic  Impacts 

As  shown  in  Table  6.1  and  Figure  6.4,  the  Texas  ERA  program  was  the  only  one  to 
produce  increases  in  employment  retention  and  earnings  for  unemployed  TANF  recipients. 
Among  the  three  Texas  ERA  sites,  the  Corpus  Christi  program  had  the  most  consistent  impacts 
on  employment  retention  and  earnings.  There,  ERA  increased  average  annual  earnings  by  $640 
over  the  four-year  follow-up  period,  or  by  almost  15  percent  of  control  group  earnings.  In 
addition,  the  program  generated  its  largest  effects  on  earnings  in  the  fourth  year  of  follow-up, 
when  it  increased  earnings  by  $900,  or  by  18  percent  of  the  control  group  level.  This  suggests 
that  the  impacts  may  continue  in  the  longer  term.  The  Texas  ERA  program  in  Fort  Worth  also 
produced  increases  in  these  measures,  but  the  impacts  were  concentrated  in  Years  2 and  3 of 
follow-up.  In  Fort  Worth,  ERA  increased  earnings  in  the  third  year  of  follow-up  by  $900,  or  by 
17  percent  of  the  control  group  level,  but  the  effects  were  not  persistent  enough  to  lead  to 
positive  impacts  over  the  cumulative  follow-up  period.  The  implementation  of  the  Texas 
program  in  Fort  Worth  started  out  weak  and  improved  over  time,  which  may  have  diluted  the 
strength  of  impacts  there  because  many  program  group  members  went  through  the  program 
before  implementation  had  improved.  There  is  evidence  in  both  Corpus  Christi  and  Fort  Worth 
that  the  ERA  program  led  to  advancement  gains:  the  earnings  impacts  are  larger  in  percentage 
terms  than  the  employment  impacts,  suggesting  that  ERA  increased  hours  or  weeks  worked  or 
wage  rates,  relative  to  the  control  group  levels.  The  ERA  programs  in  Houston,  Los  Angeles 
EJC,  and  Salem  did  not  improve  employment  retention  or  advancement  outcomes. 

Programs  Serving  Employed  TANF  Recipients 

As  discussed  in  Chapter  4,  the  Chicago,  Los  Angeles  Reach  for  Success  (RFS),  River- 
side Work  Plus,  and  Riverside  Training  Focused  ERA  programs  served  employed  TANF 
recipients.3 


3The  Riverside  Work  Plus  and  Training  Focused  programs,  together,  are  referred  to  as  the  Riverside 
“Phase  2”  programs. 
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Unemployed  TANF  recipients  Employed  TANF  recipients  Employed  non-TANF  recipients 


(continued) 
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Figure  6.2  (continued) 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  See  site  summary  tables  for  the  supporting  numbers  for  these  figures. 

A two-tailed  t-test  was  applied  to  differences  between  outcomes  for  the  program  and  control  groups. 
Statistical  significance  levels  are  indicated  as:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  are  all  part  of  the  Texas  ERA  model. 

The  outcome  measures  are  discussed  in  Chapter  2. 

The  survey  samples  are  small  in  Fort  Worth  and  Riverside  PASS,  and  some  response  bias  was  noted 
in  Riverside  PASS  and  Medford.  See  Appendix  D. 

aThis  measure  includes  participation  in  any  of  the  following  activities:  English  as  a Second  Language 
(ESL)  instruction,  adult  basic  education  (ABE)  or  General  Educational  Development  (GED)  classes, 
college  courses,  or  vocational  training. 


Service  Strategies 

All  the  ERA  programs  that  served  this  target  group  prioritized  advancement.  These 
programs  provided  individualized  services  in  the  form  of  in-depth  client  assessments;  flexibility 
in  hours  and  location  of  services;  and  intensive,  ongoing  staff-client  contact.  Advancement 
services  included  career  and  job  development  activities,  education  and  training  referrals  and/or 
incentives,  and  staff  counseling  on  job-related  issues.  The  relative  emphasis  placed  on  different 
routes  toward  advancement  varied  across  programs;  some  programs  prioritized  advancement 
through  building  up  work  experience,  and  others  sought  to  foster  advancement  by  increasing 
skills  through  education  and  training. 

Contact  and  Engagement 

The  middle  set  of  bars  in  Figure  6. 1 shows  that  while  three  of  the  ERA  programs  serv- 
ing employed  TANF  recipients  increased  the  percentage  of  people  who  ever  spoke  with  staff 
from  an  employment  program,  the  percentage  of  people  with  recent  contact  — that  is,  contact 
within  four  weeks  of  responding  to  the  12-month  survey  — increased  only  in  the  Chicago  site. 
This  highlights  the  difficulty  encountered  by  staff  in  trying  to  engage  working  people  in  ERA 
services  — a pervasive  finding  in  the  ERA  field  research  (as  is  discussed  below). 

Traditional  Employment  and  Education  Services 

Control  group  members  in  this  grouping  of  ERA  tests  participated  in  job  search  and  in 
education  and  training  at  higher-than-expected  levels  (the  middle  set  of  bars  in  Figure  6.2). 
Since  all  sample  members  in  this  target  group  were  working  when  they  entered  ERA,  it  was  not 
expected  that  so  many  would  be  engaged  in  job  search.  Given  their  work  status,  it  also  was  not 
expected  that  many  sample  members  would  be  able  to  accommodate  work,  single-parenting, 
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The  Employment  Retention  and  Advancement  Project 
Figure  6.3 

Impacts  on  Retention  and  Advancement  Services 
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Figure  6.3  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  See  site  summary  tables  for  the  supporting  numbers  for  these  figures. 

A two-tailed  t-test  was  applied  to  differences  between  outcomes  for  the  program  and  control  groups. 

Statistical  significance  levels  are  indicated  as:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  are  all  part  of  the  Texas  ERA  model. 

The  outcome  measures  are  discussed  in  Chapter  2. 

The  survey  samples  are  small  in  Fort  Worth  and  Riverside  PASS,  and  some  response  bias  was  noted 
in  Riverside  PASS  and  Medford.  See  Appendix  D. 

and  participation  in  school  or  training.  Comparing  the  program  and  control  groups,  only  the 
Chicago  program  increased  levels  of  participation  in  job  search.  With  regard  to  education  and 
training,  both  of  the  Riverside  Phase  2 programs  focused  specifically  on  increasing  participation 
in  education  and  training;  only  the  Riverside  Training  Focused  program,  however,  produced  an 
increase  — a modest  one  — in  participation  in  education  and  training.4 

Other  Retention  and  Advancement  Sen/ices 

Control  group  members’  receipt  of  other  retention  and  advancement  services  was  rela- 
tively low  (the  middle  set  of  bars  in  Figure  6.3).  Nevertheless  — and  despite  a substantial 
marketing  effort  — with  the  exception  of  the  Chicago  ERA  program,  these  programs  did  not 
increase  the  take-up  of  these  services  beyond  the  levels  of  the  control  groups.  The  Chicago 
ERA  program  achieved  large  increases  (relative  to  the  control  group)  in  participation  in  em- 
ployment retention  and  advancement  services.  For  example,  nearly  30  percent  of  Chicago  ERA 
group  members  reported  that  they  received  help  finding  a better  job  while  working,  compared 
with  approximately  5 percent  of  control  group  members.  (This  is  the  highest  rate  of  receiving 
help  in  any  of  the  key  postemployment  services  across  all  the  ERA  models,  including  the  rates 
for  the  other  target  groups.) 

Economic  Impacts 

The  ERA  program  in  Chicago  was  the  only  one  serving  employed  TANF  recipients  that 
produced  increases  in  employment  retention  and  earnings.  It  also  achieved  the  largest  reduc- 
tions in  welfare  receipt  among  all  the  programs;  its  welfare  receipt  rates  were  approximately  25 


Participation  increases  relative  to  control  group  members  were  larger  among  Riverside  Training  Focused 
and  Riverside  Work  Plus  sample  members  who  were  more  disadvantaged,  including  individuals  lacking  a high 
school  diploma  or  a GED  certificate.  For  these  individuals,  increases  in  participation  in  adult  basic  education  or 
GED  classes  were  the  primary  types  of  increases. 
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The  Employment  Retention  and  Advancement  Project 
Figure  6.4 

Impacts  on  Average  Quarterly  Employment  and  Earnings 


Average  quarterly  employment  (%) 

H Impacts  ■ Control  group  levels 


Unemployed  TANF  recipients  Employed  TANF  recipients  Employed  non-TANF  recipients 


(continued) 
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Figure  6.4  (continued) 

SOURCES:  MDRC  calculations  from  the  state  administrative  records. 

NOTES:  See  site  summary  tables  for  the  supporting  numbers  for  these  figures. 

A two-tailed  t-test  was  applied  to  differences  between  outcomes  for  the  program  and  control  groups. 
Statistical  significance  levels  are  indicated  as:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

The  Corpus  Christi,  Fort  Worth,  and  Houston  sites  are  all  part  of  the  Texas  ERA  model. 

The  outcome  measures  are  discussed  in  Chapter  2. 


percent  lower  than  control  group  levels.  While  the  Chicago  ERA  program  raised  average  annual 
earnings  by  almost  $500  over  the  cumulative  four-year  follow-up  period,  or  by  7 percent  of  the 
control  group  level,  these  effects  weakened  over  time.  Impacts  on  employment  were  no  longer 
statistically  significant  during  the  whole  of  follow-up  Year  4,  but  positive  earnings  impacts 
persisted  in  some  quarters,  suggesting  that  the  program  may  have  had  a long-term  effect  on 
advancement  (Table  6.1  and  Figure  6.4). 

Programs  Serving  Individuals  Who  Were  Employed  and  Not  Receiving 
TANF 

As  discussed  in  Chapter  5,  the  Cleveland,  Eugene,  Medford,  Riverside  Post-Assistance 
Self-Sufficiency  (PASS),  and  South  Carolina  ERA  programs  served  employed  non-TANF 
recipients. 

Service  Strategies 

The  ERA  programs  that  served  this  target  group  prioritized  both  employment  retention 
and  advancement.  Since  none  of  these  programs  served  current  TANF  recipients,  participation 
was  not  mandatory  in  any  type  of  services,  and  all  the  programs  used  intensive  outreach  and 
marketing  strategies.  Most  programs  emphasized  specializing  services  to  clients’  needs  and 
interests  as  well  as  flexibility  in  the  scheduling  and  location  of  staff-client  meetings.  Retention 
services  often  included  reemployment  services  and  staff  counseling  on  job-related  issues,  in 
addition  to  help  with  traditional  work  supports  and  social  service  referrals.  Advancement 
services  for  this  target  group  included  career  counseling  and  referrals  to  education  and  training 
services  — except  in  Cleveland.  The  Cleveland  ERA  program  focused  on  employment  reten- 
tion, offered  services  at  work  sites,  and  provided  services  to  low-wage  workers  as  well  as 
trainings  for  the  supervisors  of  low-wage  workers. 
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The  Employment  Retention  and  Advancement  Project 
Table  6.1 

ERA  Models:  Summary  of  Economic  Impact  Results 


Average  Quarterly 
Employment 

Employed  4 Consecutive 
Quarters 

Average  Annual  Earnings 

Model 

Y1 

Cumulative 
Y2  Y3  Y4  follow-up 

Y1  Y2 

Cumulative 
Y3  Y4  follow-up 

Y1  Y2  Y3 

Y4 

Cumulative 

follow-up 

Not  employed  and  receivine  TANF 

Texas 

Corpus  Christi 

• 

• • • 

• 

• • 

• • • 

• 

• 

Fort  Worth 

• 

• 

• 

Houston 

Los  Angeles  EJC 

— 

• 

— 

— 

Salem 

— 

— 

— 

Employed  and  receivine  TANF 

Chicago 

• 

• • 

• 

• 

• 

Los  Angeles  RFS 

— 

— 

— 

Riverside  Phase  2 

Riverside  Training  Focused3 

Riverside  Work  Plus 

Emnloved  and  not  receivine  TANF 

Cleveland 

— 

— 

— 

Eugene3 

— 

— 

— 

Medford 

— 

— 

— 

Riverside  PASS 

• 

• • • • 

• 

• • 

• • • 

• 

• 

South  Carolina 

(continued) 
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Table  6.1  (continued) 


SOURCES:  MDRC  calculations  from  the  state  administrative  records. 

NOTES:  A dash  indicates  “not  available”  or  that  the  sample  size  is  too  small  to  yield  meaningful  results. 

Bullets  indicate  positive  statistically  significant  differences  between  outcomes  for  the  program  and  control  groups  up  to  the  10  percent  level. 
aThe  negative  impacts  for  the  Eugene  and  Riverside  Training  Focused  programs  are  not  shown  on  this  table.  See  site-specific  summary  tables  for 
negative  impacts. 


Contact  and  Engagement 

The  set  of  bars  to  the  right  in  Figure  6. 1 shows  that  three  of  the  five  ERA  programs  that 
served  employed  non-TANF  recipients  increased  the  percentage  of  individuals  ever  contacted 
by  employment  program  staff.  Yet  only  two  ERA  programs  (those  in  Cleveland  and  South 
Carolina)  increased  ongoing  contacts,  and  the  increases  were  small.5 

Traditional  Employment  and  Education  Services 

In  all  the  ERA  programs  serving  this  target  group,  increases  in  job  search  or  education 
and  training  were  either  modest  or  nonexistent  (the  set  of  bars  to  the  right  in  Figure  6.2).  There 
were  modest  increases  in  job  search  in  the  Cleveland  and  Eugene  programs  and  modest  increas- 
es in  education  and  training  in  the  Medford  and  South  Carolina  programs.  Thus,  as  was  found 
for  programs  serving  the  other  two  target  groups,  participation  increases  in  these  types  of 
services  among  employed  non-TANF  recipients  — beyond  what  control  group  members  were 
already  doing  on  their  own  — were  small. 

Other  Retention  and  Advancement  Sen/ices 

Most  programs  serving  this  target  group  increased  participation  in  other  employment 
retention  or  advancement  services  above  the  levels  of  the  control  group  (the  set  of  bars  to  the 
right  in  Figure  6.3).  However,  the  impacts  were  never  larger  than  10  percentage  points.  In 
addition,  four  of  the  five  programs  produced  increases  in  career  assessment  — a much  more 
consistent  finding  than  for  the  other  target  groups  (not  shown). 

Economic  Impacts 

The  Riverside  PASS  ERA  program  produced  large  and  consistent  increases  in  em- 
ployment retention  and  earnings  (Table  6.1  and  Figure  6.4).  The  program  also  appears  to  have 
generated  gains  in  advancement,  relative  to  the  control  group.  It  increased  average  annual 
earnings  by  $870  over  the  four-year  follow-up  period,  or  by  10  percent  of  the  control  group 
level.  In  addition,  the  program  generated  its  largest  effects  on  earnings  ($970)  in  the  fourth  year 
of  follow-up,  suggesting  the  possibility  that  the  Riverside  PASS  ERA  program  may  lead  to 
even  longer-term  earnings  gains.  Other  programs  serving  this  target  group  did  not  produce 


Although  the  engagement  and,  as  discussed  below,  participation  impacts  are  not  large  for  the  Riverside 
PASS  program,  it  is  important  to  consider  that  survey-based  participation  data  for  that  program  were  collected 
for  a small  sample  for  which  the  program  did  not  achieve  economic  impacts.  In  general,  data  from  the 
program-maintained  management  information  system  suggest  that  the  participation  impacts  for  the  Riverside 
PASS  program  may  have  been  larger  than  what  is  indicated  by  12-month  survey  data. 
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statistically  significant  positive  impacts  on  the  main  measures  of  employment  retention  or 
advancement. 


Cross-Site  Summary 

This  section  summarizes  the  common  themes  in  the  implementation,  participation,  and 
economic  impacts  analysis  across  the  three  target  groups. 

Implementation  Findings 

• Engaging  individuals  in  employment  and  retention  services  was  a consis- 
tent challenge. 

Staff  in  the  ERA  programs  expended  considerable  energy  trying  to  engage  individuals 
in  program  activities.  Many  ERA  programs,  particularly  those  targeting  individuals  outside  the 
TANF  system,  included  intensive  marketing  and  outreach  strategies.  The  ERA  programs  also 
used  a variety  of  strategies  to  maintain  staff-client  relationships  and  encourage  ongoing  partici- 
pation in  program  activities,  including,  in  some  programs,  offering  financial  incentives  to 
encourage  contact  and  working  with  individuals  at  their  workplaces.  However,  while  most 
program  group  members  reported  having  at  least  one  contact  with  ERA  program  staff,  main- 
taining contact  over  the  course  of  the  first  year  following  random  assigmnent  was  less  common. 
Only  five  programs  increased  the  percentage  of  program  group  members,  compared  with 
control  group  members,  who  were  having  contact  with  staff  from  an  employment  program  as  of 
the  end  of  the  first  year  of  follow-up.  Moreover,  even  in  these  five  programs,  increases  in 
ongoing  engagement  were  modest. 

• Rates  of  job  loss  were  very  high,  and  job  loss  occurred  quickly  in  all  the 
ERA  programs,  requiring  staff  to  spend  a significant  amount  of  their 
time  providing  reemployment  services. 

In  a number  of  the  ERA  programs  that  attempted  to  provide  services  to  individuals 
when  they  were  employed  (referred  to  as  “postemployment  programs”  in  this  report),  high 
levels  and  fast  rates  of  job  loss  had  several  implications.  One  implication  was  that  staff  had  less 
time  available  to  work  with  individuals  on  advancement  services.  Another  implication  was  that 
the  volume  of  demands  for  reemployment  services  disrupted  programs’  staffing  plans.  Staff  in 
postemployment  programs  were  prepared  for  and  anticipated  working  with  employed  clients  to 
aid  their  placement  into  better  jobs  or  ones  positioned  for  advancement.  Staff  were  also  pre- 
pared to  provide  employment  retention  services  to  attempt  to  prevent  job  loss.  In  practice,  many 
individuals  had  lost  their  jobs  by  the  time  program  staff  first  made  contact  with  them,  forcing 
staff  to  focus  on  immediate  placement  needs  instead. 
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• The  staffing  of  the  ERA  programs  was  critical,  as  most  services  hinged 
on  staff  actions  and  many  welfare  agency  staff  lacked  skills  and  expe- 
rience relevant  to  retention  and  advancement  aims. 

In  most  of  the  ERA  programs,  staff  were  responsible  for  providing  a range  of  services, 
rather  than  specializing  in  one  or  a few.  Many  of  these  services  — such  as  recruitment,  career 
counseling,  labor  market  assessment,  and  job  development  — were  not  typically  provided  in 
standard  welfare-to-work  programs.  Thus,  the  ERA  programs  often  demanded  skill  sets  and 
facilities  in  arenas  or  with  populations  that  were  new  to  staff.  Despite  efforts  to  train  and  hire 
qualified  staff,  providing  the  various  retention  and  advancement  services  was  a struggle  for 
many  ERA  program  staff. 

Participation  Findings 

• Increases  in  participation  beyond  control  group  levels  were  not  consis- 
tent or  large,  which  may  have  made  it  difficult  for  the  programs  to 
achieve  impacts  on  employment  retention  and  advancement. 

Overall,  most  of  the  ERA  programs  produced  relatively  small  impacts  on  participation 
in  program  activities  and  on  the  receipt  of  various  types  of  retention  and  advancement  assis- 
tance, but  the  reasons  for  this  differed  by  the  type  of  activity  or  “help”  provided  in  the  pro- 
grams. Many  of  the  control  groups  in  this  study  were  eligible  for  welfare-to-work  programs  that 
often  provided  such  services  as  job  search  and  education  and  training  referrals.  Therefore,  the 
control  group  levels  of  participation  in  these  services  were  often  high  (with  participation  levels 
in  education  and  training  being  unexpectedly  high  in  the  case  of  programs  serving  employed 
individuals),  and  few  ERA  programs  significantly  raised  participation  above  these  levels. 

In  addition  to  encouraging  participation  in  job  search  and  education  and  training,  most 
ERA  programs  offered  other  types  of  retention  and  advancement  assistance,  such  as  help 
resolving  problems  on  the  job  or  help  finding  a better  job  while  working.  While  a 12-month 
client  survey  showed  that  few  control  group  members  received  these  types  of  assistance,  few 
ERA  programs  increased  the  levels  of  receipt  of  these  services  by  a substantial  amount  above 
the  control  group  levels. 

While  the  impacts  on  participation  in  ERA  services  in  a number  of  program  tests  were 
infrequent  and  small  for  two  different  reasons  — high  levels  of  control  group  participation  in 
some  activities  and  low  levels  of  program  group  participation  in  others  — the  end  result  was  the 
same:  the  little  impact  on  participation  in  services  in  these  tests  made  it  unlikely  that  these 
particular  programs  would  improve  economic  outcomes  (though  there  is  no  guarantee  that  these 
programs  would  have  improved  economic  outcomes  even  if  they  had  been  able  to  boost 
participation  more  substantially).  A few  programs,  however,  did  produce  some  sizable  partici- 
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pation  impacts  and  also  increased  employment  retention  and  advancement,  as  discussed  in  more 
detail  below. 

Effectiveness  Findings 

• Out  of  the  twelve  ERA  programs  included  in  the  report,  three  programs 
— one  in  each  target  grouping  — produced  positive  economic  impacts: 
the  Texas  (Corpus  Christi  and  Fort  Worth  sites),  Chicago,  and  Riverside 
PASS  ERA  programs. 

Table  6.1  indicates  which  ERA  programs  produced  impacts  on  three  key  outcomes:  av- 
erage quarterly  employment,  working  four  consecutive  quarters,  and  total  earnings.  (Survey- 
based  impacts  are  not  shown  because  they  cover  only  one  year  of  follow-up  for  all  12  models 
and  because  longer-term  survey  data  are  available  only  for  three  programs.)  Increases  in 
employment  retention  and  earnings  were  largest  and  most  consistent  over  time  in  the  Texas 
ERA  program  in  Corpus  Christi,  in  the  Chicago  ERA  program,  and  in  the  Riverside  PASS  ERA 
program.  The  Texas  ERA  program  in  Fort  Worth  improved  over  time  and  produced  gains  in 
Years  2 and  3.  Each  of  these  three  ERA  programs  served  a different  target  group,  which 
suggests  that  employment  retention  and  advancement  programs  can  work  for  a range  of 
populations. 

• In  the  Chicago  and  Riverside  PASS  ERA  programs,  which  provided 
services  to  employed  individuals,  impacts  were  driven  by  entering 
another  job  after  random  assignment  — not  by  staying  stably  employed 
at  the  original  job. 

Of  the  three  programs  that  produced  positive  impacts,  two  of  them  initiated  services  af- 
ter people  had  found  jobs  (postemployment  programs),  and  one  of  them  (referred  to  here  as  a 
“preemployment  program”)  initiated  services  before  people  found  jobs.  It  might  be  expected 
that  the  postemployment  programs  would  achieve  their  impacts  by  lengthening  the  amount  of 
time  that  people  stayed  in  the  job  they  held  when  they  entered  the  program.  However,  in  both  of 
these  postemployment  programs,  the  impacts  reflect  participants’  entering  new  jobs  at  greater 
rates  than  control  group  members,  either  directly  from  another  job  or  after  a period  of  unem- 
ployment. 

This  result  suggests  that  even  programs  designed  to  affect  employment  retention  and 
advancement  among  those  already  working  should  be  very  deliberate  about  how  job  change  and 
job  loss  are  addressed.  Job  changes  and  reemployment  needs  may  present  opportunities  to 
improve  outcomes.  Job  losses,  for  example,  can  be  analyzed  to  understand  the  reasons  why 
people  lost  their  jobs  and  can  lead  to  developing  plans  to  avoid  job  loss  in  the  future.  Similarly, 
job  changes  can  be  used  as  a way  to  place  individuals  into  better  jobs.  In  addition,  this  result 
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suggests  that  a focus  on  employment  retention,  rather  than  job  retention,  may  be  more  effective 
or  more  realistic. 


Conclusions 

This  section  makes  several  points  regarding  the  effectiveness  of  the  ERA  programs  in 
increasing  employment  retention  and  advancement  goals.  Then,  reflections  from  the  implemen- 
tation analysis  are  presented. 

• The  results  revealed  some  strategies  that  can  be  effective  in  promoting 
employment  retention  and  advancement  for  welfare  recipients  and  oth- 
er low-income  groups. 

Table  6.2  highlights  the  features  of  these  three  programs.  Caution  should  be  exercised 
when  trying  to  identify  promising  program  features,  however,  as  some  of  the  programs  lacking 
positive  impacts  also  contained  some  of  these  features  and  it  is  not  clear  which  features  “drove” 
the  impacts  in  any  given  site.  Two  findings  from  the  research  literature  can  help  place  these 
findings  in  context.  With  regard  to  the  Texas  findings,  several  studies  have  now  shown  that 
programs  that  provide  financial  incentives  to  supplement  earnings  in  combination  with  services 
can  promote  employment  retention  among  low- wage  workers.6  With  regard  to  the  Chicago 
program,  nonexperimental  work  has  suggested  that  low-wage  workers  often  advance  by 
changing  jobs,  and  that  matching  individuals  with  jobs  in  particular  finns  that  pay  higher  wages 
can  be  an  effective  strategy  to  promote  advancement. 7 The  fact  that  these  strategies  have  been 
found  to  be  effective  in  previous  research  as  well  as  in  this  study  lends  additional  evidence  that 
these  are  promising  strategies  to  promote  employment  retention  and  advancement. 

• Improving  employment  retention  and  advancement  remains  a chal- 
lenge. 

Three- fourths  of  the  ERA  programs  included  in  this  report  did  not  produce  gains  in  the 
targeted  outcomes  beyond  what  control  group  members  were  able  to  attain  on  their  own  with 
the  existing  set  of  services  and  supports.  This  suggests  that,  despite  a range  of  programs  and 
approaches  and  significant  effort  by  staff  and  program  managers,  gains  in  employment  retention 
and  advancement  are  difficult  to  attain.  In  addition,  even  among  the  programs  that  produced 
improvements  in  retention  and  advancement  outcomes,  wages  remained  very  low,  and  many  of 


6Gennetian,  Miller,  and  Smith  (2005);  Huston  et  al.  (2003);  Michalopoulos  et  al.  (2002);  Riccio  et  al. 
(2008). 

7Andersson,  Holzer,  and  Lane  (2005).  Indeed,  a recent  nonexperimental  analysis  of  the  employment  pat- 
terns of  ERA  sample  members  confirmed  that  job  change  was  associated  with  higher  wages  for  median- 
earnings  ample  members  (Miller,  Deitch,  and  Hill,  2009). 
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The  Employment  Retention  and  Advancement  Project 
Table  6.2 

Highlights  of  Programs  That  Increased  Employment  Retention  and  Advancement 


Corpus  Christi,  Texas 
Fort  Worth,  Texas 

Chicago 

Riverside  Pass 

Strategy 

Placement,  retention,  and 
advancement:  job  search 
assistance,  stipend  for 
employed  fonner  TANF 
recipients,  reemployment 
assistance,  and  work  site 
visits 

Advancement:  job  search 
assistance,  career  counseling, 
and  reemployment  assistance 

Retention  and  advancement: 
reemployment  assistance, 
career  counseling,  and 
referrals  to  education  and 
training 

Target  population 

Unemployed  TANF 
applicants  and  recipients 

TANF  recipients  who  had 
worked  at  least  30  hours  per 
week  for  at  least  6 
consecutive  months 

Employed  fonner  TANF 
recipients  who  recently  left 
TANF 

Service  providers 

Local  workforce 
development  boards  under 
contract  with  nonprofit 
organizations 

Experienced,  for-profit, 
employment  intermediary 

Primarily  community-based 
organizations  and  a 
community  college 

Control  services 

Relatively  strong  welfare-to- 
work  program 

Standard  welfare-to-work 
program 

Limited  postemployment 
services 

Participation 

highlights 

In  Corpus  Christi,  30  percent 
took  up  the  financial 
incentive;  20  percent  did  so 
in  the  other  Texas  sites. 
Increased  percentages 
receiving  help  with  retention 
and  advancement  in  Corpus 
Christi  and  Fort  Worth. 

Increased  percentages 
receiving  help  in  finding  a 
better  job  while  working  and 
in  getting  other  forms  of 
retention  and  advancement 
help. 

While  increases  in 
participation  were  not  large, 
participation  data  are  for  a 
cohort  that  had  few  positive 
economic  impacts.3 

Economic  impacts 

Increased  employment 
retention  and  advancement  in 
Corpus  Christi  and  Fort 
Worth  but  not  in  Houston 

Increased  employment 
retention  and  advancement 
and  reduced  welfare  receipt 

Increased  employment 
retention  and  advancement 

SOURCES:  ERA  12-Month  Survey,  administrative  records,  and  interim  reports. 


NOTES:  “Participation  impacts  in  this  report  are  based  on  client  survey  data.  As  detailed  in  the  text,  survey 
response  issues  were  present  in  the  Riverside  PASS  site.  Analysis  of  program  data  suggested  that  only  the 
Riverside  PASS  program  may  have  increased  the  use  of  a broader  range  of  services.  These  data  are  not  used 
in  this  report  because  they  only  capture  services  reported  to  the  welfare  department  by  ERA  service 
providers. 
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these  families  remained  in  poverty  at  the  end  of  the  follow-up  period.  It  is  also  important  to 
consider,  however,  that  ERA  was  a test  of  multiple  strategies  rather  than  a demonstration  of  a 
few  promising  programs.  Chapter  1 notes  that  before  the  ERA  project  began,  there  was  not 
much  evidence  of  programs  that  could  improve  employment  retention  and  advancement 
outcomes  for  current  or  fonner  welfare  recipients.  The  ERA  evaluation  has  revealed  three 
distinct  approaches  that  seem  promising  and  worthy  of  further  exploration. 

* * * 

While  this  report  presents  the  final  impact  estimates  of  many  of  the  ERA  programs,  it  is 
not  the  final  word  from  the  ERA  project.  Further  analysis  is  being  conducted  of  the  rich  ERA 
databases  to  fry  to  shed  light  on  which  program  features,  participant  characteristics,  institutional 
arrangements,  or  economic  contexts  were  most  associated  with  improvements  in  employment 
retention  and  advancement.  In  addition,  other  publications  will  consider  the  costs  and  benefits 
of  selected  ERA  programs,  the  employment  and  training  dynamics  of  the  ERA  sample,  and  the 
employment  patterns  of  ERA  sample  members  in  two-parent  families.  A series  of  practice 
briefs  will  also  probe  more  deeply  and  extract  lessons  for  those  who  operate  and  staff  retention 
and  advancement  programs.  Finally,  a synthesis  document  will  consider  the  policy,  practice, 
and  research  lessons  learned  from  the  whole  of  the  ERA  project. 
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Appendix  A 

Supplementary  Exhibits  for  Chapter  3: 
Programs  Serving  Unemployed  TANF  Recipients 


Appendix  A presents  supplementary  boxes  and  tables  relating  to  the  Texas,  Los  Angeles  EJC, 
and  Salem  ERA  tests. 

Box 

A.  1 Program  Summary:  The  Texas  ERA  Program  in  Corpus  Christi,  Fort  Worth,  and  Houston 
A.2  Program  Summary:  Los  Angeles  EJC 

A.3  Program  Summary:  Salem 

Table 

A.l  Impacts  on  Contacts,  Services,  and  Participation  at  the  12 -Month  Survey:  Coipus  Christi 

A.2  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Coipus  Christi 

A.3  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4):  Coipus  Christi 

A.4  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Coipus  Christi 

A.  5 Impacts  on  Contacts,  Services,  and  Participation  at  the  12 -Month  Survey:  Fort  Worth 

A.  6 Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 

Period  (Years  1-4):  Fort  Worth 

A.7  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Fort  Worth 
A.8  Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Houston 
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Table 

A.9  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 
Houston 

A.  1 0 Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 

Period  (Years  1-4):  Houston 

A.  1 1 Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Houston 

A.  12  Impacts  on  Contacts,  Services,  and  Participation  at  the  12 -Month  Survey: 

Los  Angeles  EJC 

A.  13  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Los  Angeles  EJC 

A.  14  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-3):  Los  Angeles  EJC 

A.  15  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Los  Angeles  EJC 

A.  16  Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Salem 

A.  17  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income:  Salem 

A.  1 8 Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 

Period  (Years  1-3):  Salem 

A.  19  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Salem 
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Appendix  Box  A.l 


Program  Summary: 

The  Texas  ERA  Program  in  Corpus  Christi,  Fort  Worth,  and  Houston 

“Combining  financial  incentives  with  postemployment  services  for  work  and  retention” 

Goal:  Job  placement,  job  retention,  and  career  advancement  for  unemployed  TANF  applicants 
and  recipients 

Locations:  Corpus  Christi,  Fort  Worth,  and  Houston 

Target  population:  Unemployed  TANF  applicants  and  recipients 

Implementation  schedule:  Random  assignment  occurred  in  Corpus  Christi  and  Fort  Worth 
from  October  2000  to  January  2003  and  in  Houston  from  March  2001  to  December  2002.  All 
sites  operated  the  ERA  program  from  the  beginning  of  random  assignment  to  August  2004. 

Management  structure:  Operated  by  local  nonprofit  organizations  contracted  by  Local 
Workforce  Development  Boards 

Participation  requirements:  Preemployment  programs  were  mandatory  for  individuals  while 
they  were  receiving  TANF;  however,  participation  was  not  required  after  leaving  TANF.  In 
order  to  receive  the  stipend,  individuals  were  required  to  exhaust  their  TANF  earnings 
disregard,  work  30  hours  per  week,  submit  necessary  documentation,  and  attend  some  type  of 
employment-related  activity  each  month. 

Outreach  and  marketing:  Due  to  low  initial  take-up  of  the  stipend,  staff  developed  new 
marketing  strategies.  Stipend  marketing  was  strong  in  Corpus  Christi,  was  initially  limited  in 
Fort  Worth,  and  was  very  weak  until  late  in  Houston. 

Staff-client  engagement: 

• The  staff-client  relationship  was  initiated  during  preemployment  services  and  continued 
after  individuals  found  employment. 

• Services  were  tailored  to  individual  participants,  with  limited  flexibility  in  meeting 
times  and  locations. 

• Client  assessment  was  aimed  toward  development  of  career  plans. 

Initial  job  preparation  and  placement  services:  One -week  job  search  workshop,  including 
job  search  resources,  resume  development,  and  interviewing  techniques,  followed  by  three  to 
five  weeks  of  directed  individual  job  search 

Retention  services: 

• Individualized  assistance  with  job-related  issues 

• Monthly  stipend  of  $200  for  former  TANF  recipients  working  at  least  30  hours  per 
week 

Advancement  services:  Corpus  Christi  staff  discussed  participants’  career  paths  and  the 
requirements  to  obtain  the  ideal  job,  and  they  made  site  visits  to  employers.  Similar  services 
developed  over  time  in  Fort  Worth  and  Houston. 

(continued) 


253 


Appendix  Box  A.1  (continued) 


Employer  linkages:  Strong  in  Corpus  Christi,  gained  strength  in  Fort  Worth,  and  limited  in 
Houston 

Key  funding  source:  State  TANF  program;  stipend  funded  by  AFDC  sanction  resettlement 
funds 

Implementation  challenges: 

• All  the  sites  faced  staffing  shortages  and  caseloads  that  were  higher  than  intended. 

• Stipend  receipt  rates  were  lower  than  expected. 
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Appendix  Box  A.2 


Program  Summary: 

Los  Angeles  EJC  (Enhanced  Job  Club) 

“A  mandatory,  enhanced  job  club  focused  on  finding  jobs  within  careers  of  interest” 

Goal:  Place  unemployed  TANF  recipients  in  targeted  and  promotable  jobs  within  an  identified 
field  of  interest 

Locations:  San  Gabriel  Valley  region  and  Central  Los  Angeles  region  of  Los  Angeles  County 
Target  population:  Unemployed  TANF  recipients 

Implementation  schedule:  Random  assignment  occurred  from  June  2002  through  September 
2004.  Job  clubs  ran  from  the  beginning  of  random  assignment  through  December  2004. 

Management  structure:  Jointly  operated  by  the  Los  Angeles  County  Department  of  Public 
Social  Services  and  the  Los  Angeles  County  Office  of  Education 

Participation  requirements:  Mandatory  participation  in  job  club  workshops  in  order  to 
continue  receiving  the  TANF  grant 

Outreach  and  marketing:  Because  participation  was  mandatory,  staff  did  not  recruit 
individuals  or  attempt  to  engage  participants  prior  to  the  scheduled  start  date  of  their  job  club. 

Staff-client  engagement: 

• Workshops  were  conducted  during  standard  office  hours  at  the  program  workshop 
offices. 

• Ongoing  staff-client  relationships  were  not  a priority  of  the  model,  and  staff  did  not 
attempt  to  stay  in  contact  with  participants  (employed  or  not)  after  they  left  the  job  club. 

Initial  job  preparation  and  placement  services:  Job  search  activities  were  organized  as  a 
step-down  approach,  targeting  a specific  range  of  jobs  within  an  occupational  field  of  interest 
defined  by  the  participant. 

Retention  services:  Did  not  provide  specific  services  in  this  area  but  addressed  it  in  indirect 
ways,  such  as  encouraging  participants  to  meet  employers’  expectations  for  work  site  behavior 
and  ensuring  that  participants  made  arrangements  for  child  care  and  transportation  needs  in 
preparation  for  employment. 

Advancement  services: 

• Career  development  activities  — formulation  of  a career  plan  and  a list  of  targeted  and 
promotable  jobs  that  participants  attempted  to  find 

• Quick  access  to  education  and  training  if  unable  to  find  suitable  employment 

• Encouraged  blending  part-time  work  with  participation  in  education  or  training 

Employer  linkages:  Minimal  strategies  to  involve  employers  in  job  placement  or  career 
development  activities 

Key  funding  source:  State  TANF  program 

Implementation  challenges:  Implemented  largely  as  designed 
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Appendix  Box  A.3 


Program  Summary: 

Salem 

“Combining  pre-  and  postemployment  services  for  success  in  a field  of  interest” 

Goal:  Job  placement,  job  retention,  and  career  advancement  among  unemployed  TANF 
applicants 

Location:  Salem,  Oregon 

Target  population:  Unemployed  TANF  applicants 

Implementation  schedule:  Random  assignment  occurred  from  May  2002  to  May  2004,  and  the 
program  operated  from  the  beginning  of  random  assignment  to  June  2005. 

Management  structure:  Jointly  operated  by  the  welfare  agency  (DHS)  and  Chemeketa 
Community  College.  Staff  were  colocated  at  the  Winema  Career  Center  (a  W1A  One-Stop 
Career  Center). 

Participation  requirements:  Mandatory  participation  in  the  preemployment  j ob  search  to 
continue  to  receive  TANF  grant;  postemployment  services  were  voluntary. 

Outreach  and  marketing:  Staff  conducted  intake  meetings  and  orientation  sessions  with  the 
marketing  of  ERA  program  services,  emphasizing  the  program’s  enhanced  customer  service  and 
ongoing  postemployment  services. 

Staff-client  engagement:  Services  were  tailored  to  individual  participants’  interests  and 
employment  barriers,  with  flexibility  in  the  scheduling  and  location  of  staff-client  meetings. 
Frequent  contact  was  made  with  engaged  participants  through  workshops  and  regular  check-ins. 

Initial  job  preparation  and  placement  services:  Preemployment  workshops  and  one-on-one 
assistance  from  program  staff  to  address  long-term  career  interests  and  employment  barriers  and 
provide  support  services 

Retention  services:  Staff  were  unable  to  provide  postemployment  services  consistently. 

Advancement  services:  Preemployment  workshops,  including  advancement  messages,  career 
interest  assessments,  and  career  path  activities 

Employer  linkages:  Staff  were  unsure  of  how  to  approach  employers.  Participants  repeated 
that  they  did  not  want  program  staff  involved  in  their  work  relationships. 

Key  funding  source:  State  TANF  program 

Implementation  challenges: 

• Staff  lacked  experience  in  providing  retention  and  advancement  services. 

• Higher-than-anticipated  caseloads  made  it  difficult  for  staff  to  focus  on 
postemployment  services. 

• State  hiring  freeze  affected  job  opportunities  available  to  participants  as  well  as  the 
program’s  staffing. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.l 

Impacts  on  Contacts,  Services  and  Participation  at  the  12-Month  Survey 

Corpus  Christi 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

60.2 

47.7 

12.5  ** 

0.041 

Average  number  of  contacts  with  staff 

6.6 

6.4 

0.3 

0.889 

Talked  with  staff  in  past  4 weeks  (%) 

27.2 

16.5 

10.7  ** 

0.036 

Staff  ever  talked  with  respondent's  employer  (%) 

17.8 

2.1 

15  7 *** 

0.000 

Services 

Received  help  with  support  services  (%) 

43.6 

33.2 

10.4  * 

0.063 

Received  help  with  basic  needs  (%) 

38.6 

46.1 

-7.5 

0.229 

Received  help  with  public  benefits  (%) 

74.8 

71.5 

3.3 

0.548 

Received  help  with  job  preparation  (%) 

43.2 

39.6 

3.6 

0.562 

Received  help  with  retention/advancement  (%) 

28.5 

14.0 

14,6  *** 

0.003 

Finding  a better  job  while  working 

13.6 

3.9 

9 7 *** 

0.006 

Other  activities  while  working 

11.8 

2.2 

9 6 *** 

0.002 

Career  assessment 

17.8 

7.4 

10.4  ** 

0.010 

Dealing  with  problems  on  the  job 

8.4 

1.5 

6 9 *** 

0.009 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

8.1 

5.1 

3.0 

0.338 

ParticiDation 

Participated  in  any  activity  (%) 

74.0 

67.6 

6.4 

0.250 

Participated  in  any  employment-related  activity2  (%) 

62.0 

58.2 

3.8 

0.531 

Participated  in  a job  search  activity  (%) 

62.0 

58.2 

3.8 

0.531 

Group  job  search/job  club 

48.3 

42.9 

5.3 

0.382 

Individual  job  search 

49.9 

42.7 

7.2 

0.245 

Participated  in  an  education/training  activity  (%) 

30.6 

22.7 

7.9 

0.150 

ABE/GED  and  ESL 

12.0 

8.1 

3.9 

0.273 

College  courses 

15.0 

12.0 

3.1 

0.436 

Vocational  training 

5.3 

4.4 

1.0 

0.719 

Participated  in  unpaid  work/ subsidized  employment  (%) 

4.3 

1.3 

2.9 

0.157 

Participated  in  an  education  activity  while  working  (%) 

19.2 

13.4 

5.9 

0.200 

Participated  in  an  employment  activity  while  working  (%) 

17.1 

15.3 

1.8 

0.695 

Average  number  of  weeks  participating  in 

Job  search  activities 

3.9 

3.6 

0.3 

0.743 

Education/training  activities 

5.7 

5.2 

0.5 

0.760 

Unpaid  work/subsidized  employment 

0.4 

0.4 

0 

0.940 

Sample  size  (total  = 290) 

141 

149 

(continued) 
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Appendix  Table  A.l  (continued) 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  on-the-job 
training,  or  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.2 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Corpus  Christi 

ERA  Control  Difference  Standard 

Outcome  Group  Group  (Impact)  P-Value  Error 

Follow-Up  Year  1 


Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 

Follow-Up  Year  2 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 

Follow-Up  Year  3 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 

Follow-Up  Year  4 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income3  ($) 


72.7 

73.8 

-1.0 

0.610 

2.0 

52.5 

49.6 

2.9  * 

0.099 

1.7 

30.0 

26.1 

3.9  * 

0.057 

2.0 

11.7 

10.7 

1.0 

0.486 

1.4 

3,990 

3,563 

428  ** 

0.044 

212 

81.8 

78.3 

3.6  * 

0.061 

1.9 

95.9 

95.9 

0.0 

0.969 

0.9 

7,758 

7,413 

344  * 

0.089 

202 

70.3 

68.2 

2.1 

0.325 

2.1 

52.1 

49.2 

2.9 

0.117 

1.8 

33.2 

31.1 

2.0 

0.333 

2.1 

17.7 

15.5 

2.2 

0.184 

1.7 

4,738 

4,279 

460  * 

0.086 

267 

37.4 

43.4 

-6.0  *** 

0.008 

2.3 

82.8 

85.1 

-2.3 

0.173 

1.7 

7,655 

7,455 

200 

0.440 

259 

69.1 

63.7 

5 5** 

0.011 

2.2 

50.6 

46.5 

4.1  ** 

0.029 

1.9 

32.2 

29.8 

2.4 

0.251 

2.1 

23.0 

17.5 

5 5 *** 

0.002 

1.8 

5,399 

4,624 

775  ** 

0.014 

316 

24.3 

28.1 

bo 

* 

0.064 

2.1 

77.1 

79.4 

-2.2 

0.234 

1.9 

8,183 

7,623 

560  * 

0.069 

307 

67.5 

63.0 

4.5  ** 

0.038 

2.2 

51.5 

46.7 

4.8  ** 

0.012 

1.9 

34.3 

29.8 

4.5  ** 

0.036 

2.2 

21.4 

19.4 

2.0 

0.268 

1.8 

5,915 

5,018 

896  ** 

0.013 

360 

17.7 

18.3 

-0.6 

0.741 

1.8 

75.8 

77.0 

-1.2 

0.548 

1.9 

8,699 

7,904 

795  ** 

0.025 

354 

Sample  size  (total  = 1,727)  870  857 

(continued) 
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Appendix  Table  A.2  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Total  income  does  not  include  the  value  of  the  stipend. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.3 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

Corpus  Christi 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment3 

1.8 

1.6 

0.3  * 

0.063 

Average  quarterly  employment  (%) 

51.7 

48.0 

3 7 ** 

0.012 

Percentage  of  quarters  employed  (%) 

Never  employed 

11.6 

10.7 

0.9 

0.535 

1-25 

18.7 

26.1 

_y  4 *** 

0.000 

26-50 

21.1 

18.4 

2.7 

0.156 

51-75 

19.2 

18.5 

0.7 

0.706 

76-100 

29.5 

26.4 

3.1 

0.130 

Employed  entire  follow-up  period  (%) 

11.6 

9.0 

2.6  * 

0.068 

Had  employment  spell  of  at  least  4 quarters  (%) 

63.4 

56.9 

g 5 *** 

0.003 

Average  number  of  employers  during  follow-up  period 

4.3 

4.1 

0.2 

0.201 

Number  of  employers  (%) 

Never  employed 

11.6 

10.7 

0.9 

0.535 

1 to  2 

25.9 

29.3 

-3.4 

0.112 

3 to  4 

23.7 

24.2 

-0.5 

0.802 

5 to  8 

25.2 

24.6 

0.7 

0.744 

More  than  8 

13.7 

11.3 

2.3 

0.135 

Average  number  of  quarters  in  first  employment  spell 

5.3 

4.8 

* 

* 

© 

0.038 

Number  of  quarters  until  first  employment  spell 

3.3 

3.1 

0.2 

0.393 

Average  number  of  employment  spells 

1.8 

1.8 

0.0 

0.922 

Average  length,  longest  employment  spell  (quarters) 

6.4 

6.0 

0 4** 

0.044 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

11.6 

10.7 

0.9 

0.535 

1 to  2 

16.6 

23.6 

_1  q *** 

0.000 

3 to  4 

16.7 

15.9 

0.8 

0.637 

5 to  8 

24.4 

21.7 

2.7 

0.179 

9 to  12 

13.2 

13.0 

0.3 

0.870 

13  to  16 

17.5 

15.2 

2.3 

0.178 

(continued) 
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Appendix  Table  A.3  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Average  number  of  unemployment  spells 

1.7 

1.8 

-0.1 

0.184 

Length  of  longest  unemployment  spell,  in  quarters 

6.1 

6.6 

-0.4  * 

0.057 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

11.6 

9.0 

2.6  * 

0.068 

1 to  2 

20.4 

18.9 

1.5 

0.416 

3 to  4 

17.6 

16.2 

1.5 

0.409 

5 to  8 

21.5 

24.1 

-2.6 

0.197 

9 to  12 

12.0 

13.5 

-1.5 

0.337 

13  to  16 

17.0 

18.4 

-1.4 

0.414 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

5,011 

4,371 

640  *** 

0.007 

Average  annual  earnings  (%) 

$0 

11.6 

10.7 

0.9 

0.535 

$l-$  1,999 

30.3 

37.4 

_7  | *** 

0.001 

$2,000-84,999 

21.7 

19.8 

1.9 

0.326 

$5,000-59,999 

20.0 

18.3 

1.7 

0.356 

$10,000-$  14,999 

8.7 

8.3 

0.4 

0.750 

$ 1 5,000-$  19,999 

5.0 

3.9 

1.2 

0.236 

$20,000  or  higher 

2.6 

1.7 

1.0 

0.134 

Average  annual  earnings  of  $10,000  or  more  (%) 

16.4 

13.8 

2.6 

0.108 

Quarters  with  earnings  of  $3,500  or  more  (%) 

12.8 

10.9 

1.9  ** 

0.045 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

11.6 

10.7 

0.9 

0.535 

No  quarters  with  earnings  above  $3,500 

50.8 

54.4 

-3.5 

0.127 

1 to  2 

14.2 

14.0 

0.2 

0.914 

3 to  4 

6.6 

5.4 

1.2 

0.309 

5 to  8 

7.1 

7.9 

-0.8 

0.523 

9 to  12 

5.4 

6.0 

-0.6 

0.611 

13  to  16 

4.3 

1.7 

2.6  *** 

0.001 

ComDarison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

15.1 

15.8 

-0.7 

0.653 

Earnings  decreased  (%) 

No  longer  employed 

17.4 

21.2 

-3.8  ** 

0.046 

Earnings  decreased  by  less  than  $250 

3.5 

2.8 

0.7 

0.419 

Earnings  decreased  by  $250  or  more 

15.8 

15.7 

0.1 

0.934 

Earnings  increased  (%) 

Became  employed 

12.2 

10.4 

1.8 

0.243 

Earnings  increased  by  less  than  $250 

3.5 

4.2 

-0.7 

0.450 

Earnings  increased  by  $250  or  more 

32.5 

29.8 

2.6 

0.224 

Average  earnings  per  quarter  employed  ($) 

2,424 

2,277 

147 

Sample  size  (total  = 1,727) 

870 

857 

(continued) 
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Appendix  Table  A.3  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups 
arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for 
sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program 
group  and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be 
attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously.  Zeroes  were  given  to  those  who 
were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.4 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Corpus  Christi 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

76.8 

70.2 

6.6 

0.227 

Currently  employed 

53.0 

47.8 

5.2 

0.399 

No  longer  employed 

23.8 

21.8 

2.1 

0.699 

Current  working  status  (%) 

Full  time 

37.3 

33.8 

3.5 

0.531 

Part  time 

15.7 

14.0 

1.6 

0.716 

Currently  employed  at  a "good  job"a(%) 

10.3 

9.7 

0.6 

0.862 

Hours 

Average  hours  per  week 

17.2 

16.5 

0.7 

0.738 

Total  hours  per  week  (%) 

Less  than  30 

15.7 

14.0 

1.6 

0.716 

30-34 

12.0 

6.8 

5.3 

0.147 

35-44 

20.3 

20.4 

-0.2 

0.974 

45  or  more 

5.1 

6.6 

-1.6 

0.574 

Earnings 

Average  hourly  wage  (%) 

Less  than  $5.00 

5.6 

8.8 

-3.2 

0.322 

$5.00  - $6.99 

23.9 

20.4 

3.5 

0.496 

$7.00  - $8.99 

14.8 

11.5 

3.4 

0.416 

$9.00  or  more 

8.7 

7.2 

1.5 

0.656 

Average  weekly  earnings  ($) 

123 

110 

13 

0.466 

Total  earnings  per  week  (%) 

Less  than  $200 

22.5 

19.7 

2.8 

0.579 

$201-$300 

20.3 

16.4 

3.9 

0.401 

$301-$500 

7.0 

10.9 

-3.9 

0.270 

$500  or  more 

3.3 

1.0 

2.3 

0.159 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

13.0 

11.9 

1.1 

0.792 

Paid  vacation 

18.6 

11.2 

7.4  * 

0.083 

Paid  holidays  other  than  Christmas  and  New  Year 

16.5 

15.3 

1.2 

0.784 

Dental  benefits 

13.2 

11.0 

2.2 

0.585 

A retirement  plan 

13.3 

10.2 

3.1 

0.429 

A health  plan  or  medical  insurance 

16.1 

12.9 

3.2 

0.446 

(continued) 
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Appendix  Table  A.4  (continued) 


Outcome 


ERA  Control  Difference 
Group  Group  (Impact) P-Value 


Schedule11  (%) 


Regular 

30.1 

27.9 

2.2 

0.690 

Split  shift 

-0.1 

1.4 

-1.5 

0.159 

Irregular 

2.2 

3.3 

-1.1 

0.588 

Evening  shift 

5.7 

7.4 

-1.7 

0.579 

Night  shift 

1.8 

3.0 

-1.2 

0.533 

Rotating  shift 

10.4 

1.5 

g q *** 

0.002 

Other  schedule 

1.1 

1.0 

0.1 

0.928 

Odd  job 

1.8 

2.4 

-0.6 

0.727 

Job  skills 

Percentage  reporting  that  job  requires  each  at  least  monthly 


Reading  and  writing  skills 

38.3 

32.9 

5.4 

0.352 

Work  with  computers 

24.8 

16.8 

7.9 

0.104 

Arithmetic  skills 

30.4 

21.6 

8.7 

0.102 

Customer  contact 

47.7 

42.8 

4.9 

0.422 

Sample  size  (total  = 290) 

141 

149 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.5 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Fort  Worth 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

63.0 

44.9 

18.1  ** 

0.014 

Average  number  of  contacts  with  staff 

14.0 

5.2 

g 9 *** 

0.007 

Talked  with  staff  in  past  4 weeks  (%) 

29.7 

9.0 

20  7 *** 

0.001 

Staff  ever  talked  with  respondent's  employer  (%) 

8.4 

1.4 

2 0** 

0.035 

Services 

Received  help  with  support  services  (%) 

54.6 

43.5 

11.2 

0.122 

Received  help  with  basic  needs  (%) 

40.6 

48.6 

-8.0 

0.309 

Received  help  with  public  benefits  (%) 

66.0 

71.1 

-5.1 

0.478 

Received  help  with  job  preparation  (%) 

52.1 

42.8 

9.4 

0.231 

Received  help  with  retention/advancement  (%) 

31.0 

10.9 

20.1  *** 

0.002 

Finding  a better  job  while  working 

13.5 

2.7 

10.7  ** 

0.012 

Other  activities  while  working 

13.3 

2.9 

10.4  ** 

0.015 

Career  assessment 

20.9 

6.0 

14  9 *** 

0.005 

Dealing  with  problems  on  the  job 

5.8 

1.7 

4.1 

0.177 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

5.4 

3.2 

2.2 

0.480 

ParticiDation 

Participated  in  any  activity  (%) 

88.7 

77.5 

11.2  * 

0.060 

Participated  in  any  employment-related  activity3  (%) 

78.1 

74.2 

3.9 

0.570 

Participated  in  a job  search  activity  (%) 

77.4 

72.7 

4.6 

0.500 

Group  job  search/job  club 

62.7 

49.3 

13.4  * 

0.086 

Individual  job  search 

67.2 

66.9 

0.2 

0.974 

Participated  in  an  education/training  activity  (%) 

38.7 

29.5 

9.2 

0.193 

ABE/GED  and  ESL 

20.7 

16.7 

4.0 

0.464 

College  courses 

11.9 

10.5 

1.4 

0.762 

Vocational  training 

11.1 

7.0 

4.1 

0.362 

Participated  in  unpaid  work/subsidized  employment  (%) 

14.7 

11.9 

2.8 

0.595 

Participated  in  an  education  activity  while  working  (%) 

12.9 

10.6 

2.3 

0.608 

Participated  in  an  employment  activity  while  working  (%) 

20.7 

16.6 

4.1 

0.507 

Average  number  of  weeks  participating  in 

Job  search  activities 

15.3 

12.4 

2.9 

0.365 

Education/training  activities 

7.7 

6.6 

1.1 

0.634 

Unpaid  work/subsidized  employment 

2.8 

1.5 

1.2 

0.329 

Sample  size  (total  = 188) 

92 

96 

(continued) 
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Appendix  Table  A.5  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.6 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

Fort  Worth 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Emolovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment3 

1.4 

1.4 

0.1 

0.609 

Average  quarterly  employment  (%) 

48.5 

46.9 

1.6 

0.274 

Percentage  of  quarters  employed  (%) 

Never  employed 

13.1 

15.0 

-2.0 

0.226 

1-25 

21.4 

21.6 

-0.2 

0.934 

26-50 

19.5 

19.0 

0.5 

0.796 

51-75 

20.1 

19.3 

0.8 

0.692 

76-100 

26.0 

25.2 

0.8 

0.682 

Employed  entire  follow-up  period  (%) 

8.5 

8.4 

0.1 

0.943 

Had  employment  spell  of  at  least  4 quarters  (%) 

61.3 

58.5 

2.8 

0.214 

Average  number  of  employers  during  follow-up  period 

4.0 

4.1 

-0.1 

0.625 

Number  of  employers  (%) 

Never  employed 

13.1 

15.0 

-2.0 

0.226 

1 to  2 

28.7 

27.4 

1.3 

0.576 

3 to  4 

21.3 

19.0 

2.3 

0.266 

5 to  8 

25.7 

26.4 

-0.7 

0.737 

More  than  8 

11.3 

12.1 

-0.8 

0.596 

Average  number  of  quarters  in  first  employment  spell 

5.0 

4.5 

0.4  * 

0.063 

Number  of  quarters  until  first  employment  spell 

3.6 

3.9 

-0.3 

0.249 

Average  number  of  employment  spells 

1.7 

1.7 

0.0 

0.971 

Average  length,  longest  employment  spell  (quarters) 

6.1 

5.8 

0.3 

0.153 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

13.1 

15.0 

-2.0 

0.226 

1 to  2 

18.9 

18.7 

0.2 

0.918 

3 to  4 

15.5 

16.0 

-0.5 

0.792 

5 to  8 

23.4 

22.6 

0.8 

0.714 

9 to  12 

12.4 

14.2 

-1.7 

0.306 

13  to  16 

16.8 

13.5 

3.2  * 

0.063 

(continued) 
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Appendix  Table  A.6  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Average  number  of  unemployment  spells 

1.8 

1.8 

0.0 

0.654 

Length  of  longest  unemployment  spell,  in  quarters 

6.6 

6.9 

-0.3 

0.160 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

8.5 

8.4 

0.1 

0.943 

1 to  2 

20.2 

18.3 

1.9 

0.329 

3 to  4 

18.0 

18.0 

0.0 

0.996 

5 to  8 

22.1 

22.0 

0.1 

0.960 

9 to  12 

11.5 

11.7 

-0.2 

0.913 

13  to  16 

19.7 

21.6 

-1.9 

0.305 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

5,706 

5,256 

449 

0.148 

Average  annual  earnings  (%) 

$0 

13.1 

15.0 

-2.0 

0.226 

$l-$  1,999 

28.8 

29.2 

-0.4 

0.871 

$2,000-$4,999 

20.5 

20.5 

0.0 

0.986 

$5,000-$9,999 

17.4 

16.9 

0.5 

0.796 

$10,000-$14,999 

8.7 

10.2 

-1.5 

0.301 

$15, 000-$  19, 999 

6.0 

3.9 

2.2  ** 

0.048 

$20,000  or  higher 

5.4 

4.2 

1.2 

0.254 

Average  annual  earnings  of  $ 10,000  or  more  (%) 

20.2 

18.4 

1.8 

0.324 

Quarters  with  earnings  of  $3,500  or  more  (%) 

17.1 

15.5 

1.7 

0.154 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

13.1 

15.0 

-2.0 

0.226 

No  quarters  with  earnings  above  $3,500 

41.5 

39.3 

2.3 

0.343 

1 to  2 

14.8 

16.0 

-1.2 

0.512 

3 to  4 

7.9 

7.7 

0.2 

0.877 

5 to  8 

8.9 

11.7 

-2.8  * 

0.069 

9 to  12 

7.0 

5.5 

1.4 

0.232 

13  to  16 

6.8 

4.8 

2.0  * 

0.074 

Comparison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

18.0 

19.8 

-1.8 

0.336 

Earnings  decreased  (%) 

No  longer  employed 

18.1 

18.0 

0.2 

0.938 

Earnings  decreased  by  less  than  $250 

2.5 

2.3 

0.2 

0.834 

Earnings  decreased  by  $250  or  more 

14.7 

14.1 

0.7 

0.712 

Earnings  increased  (%) 

Became  employed 

13.6 

11.8 

1.8 

0.285 

Earnings  increased  by  less  than  $250 

2.6 

2.6 

0.0 

0.963 

Earnings  increased  by  $250  or  more 

30.3 

31.5 

-1.2 

0.601 

Average  earnings  per  quarter  employed  ($) 

2,943 

2,805 

138 

Sample  size  (total  = 1,572) 

784 

788 

(continued) 
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Appendix  Table  A.6  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control 
groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental. These  measures  were  computed  only  for 
sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program 
group  and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily 
be  attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously.  Zeroes  were  given  to  those 
who  were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.7 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Fort  Worth 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

EmDlovment  status 

Ever  employed  since  random  assignment  (%) 

72.1 

74.7 

-2.6 

0.708 

Currently  employed 

43.1 

38.9 

4.3 

0.580 

No  longer  employed 

29.0 

35.8 

-6.8 

0.352 

Current  working  status  (%) 

Full  time 

36.5 

31.7 

4.7 

0.525 

Part  time 

6.7 

7.1 

-0.5 

0.911 

Currently  employed  at  a "good  job"a  (%) 

15.6 

9.1 

6.5 

0.195 

Hours 

Average  hours  per  week 

16.0 

14.3 

1.7 

0.566 

Total  hours  per  week  (%) 

Less  than  30 

6.7 

7.1 

-0.5 

0.911 

30-34 

2.9 

7.7 

-4.8 

0.164 

35-44 

27.6 

17.3 

10.3 

0.123 

45  or  more 

6.0 

6.8 

-0.8 

0.840 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

5.8 

9.1 

-3.3 

0.417 

$5.00  - $6.99 

11.3 

10.0 

1.2 

0.797 

$7.00  - $8.99 

13.7 

10.8 

2.9 

0.556 

$9.00  or  more 

12.4 

9.0 

3.4 

0.460 

Average  weekly  earnings  ($) 

131 

101 

31 

0.264 

Total  earnings  per  week  (%) 

Less  than  $200 

12.0 

10.4 

1.6 

0.752 

$201-$300 

13.5 

17.3 

-3.8 

0.505 

$301-$500 

12.5 

9.9 

2.6 

0.589 

$500  or  more 

5.2 

1.3 

4.0 

0.150 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

13.0 

7.3 

5.7 

0.208 

Paid  vacation 

12.6 

10.8 

1.8 

0.716 

Paid  holidays  other  than  Christmas  and  New  Year 

10.7 

7.5 

3.2 

0.483 

Dental  benefits 

12.1 

8.2 

4.0 

0.393 

A retirement  plan 

7.2 

8.7 

-1.5 

0.722 

A health  plan  or  medical  insurance 

14.4 

9.1 

5.2 

0.296 

(continued) 
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Appendix  Table  A.7  (continued) 


Outcome 


ERA  Control  Difference 
Group  Group  (Impact) P- Value 


Schedule"  (%) 


Regular 

21.2 

21.4 

-0.2 

0.970 

Split  shift 

0.7 

2.5 

-1.8 

0.350 

Irregular 

1.6 

3.7 

-2.1 

0.382 

Evening  shift 

1.7 

4.6 

-2.9 

0.305 

Night  shift 

5.4 

1.1 

4.4 

0.114 

Rotating  shift 

7.2 

0.4 

6.9  ** 

0.019 

Other  schedule 

1.2 

-0.1 

1.3 

0.265 

Odd  job 

4.2 

5.4 

-1.2 

0.707 

Job  skills 

Percentage  reporting  that  job  requires  each  at  least  monthly 


Reading  and  writing  skills 

31.1 

25.4 

5.8 

0.395 

Work  with  computers 

18.6 

7.2 

1 1.4  ** 

0.015 

Arithmetic  skills 

23.7 

13.8 

9.9  * 

0.088 

Customer  contact 

38.3 

33.1 

5.3 

0.487 

Sample  size  (total  = 1 88) 

92 

96 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "goodjob"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.8 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Houston 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

59.5 

58.4 

1.1 

0.853 

Average  number  of  contacts  with  staff 

9.7 

6.1 

3.6 

0.118 

Talked  with  staff  in  past  4 weeks  (%) 

21.0 

20.1 

1.0 

0.836 

Staff/case  manager  ever  talked  with  respondent's  employer  (%) 

5.4 

8.9 

-3.5 

0.252 

Services 

Received  help  with  support  services  (%) 

42.7 

41.5 

1.1 

0.842 

Received  help  with  basic  needs  (%) 

43.6 

50.8 

-7.2 

0.212 

Received  help  with  public  benefits  (%) 

57.7 

71.1 

-13.4  ** 

0.017 

Received  help  with  job  preparation  (%) 

42.0 

40.8 

1.2 

0.834 

Received  help  with  retention/advancement  (%) 

24.0 

18.3 

5.7 

0.233 

Finding  a better  job  while  working 

10.0 

5.4 

4.6 

0.135 

Other  activities  while  working 

14.7 

3.4 

1 1 3 *** 

0.001 

Career  assessment 

12.4 

11.2 

1.1 

0.756 

Dealing  with  problems  on  the  job 

11.7 

6.4 

5.3 

0.111 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

7.5 

7.3 

0.1 

0.967 

ParticiDation 

Participated  in  any  activity  (%) 

82.8 

78.8 

4.1 

0.378 

Participated  in  any  employment-related  activity3  (%) 

80.1 

70.7 

9.4  * 

0.059 

Participated  in  a job  search  activity  (%) 

79.6 

69.1 

10.4  ** 

0.038 

Group  job  search/job  club 

59.8 

51.9 

7.9 

0.172 

Individual  job  search 

67.7 

59.5 

8.1 

0.141 

Participated  in  an  education/training  activity  (%) 

35.2 

34.9 

0.3 

0.960 

ABE/GED  and  ESL 

19.4 

21.7 

-2.2 

0.606 

College  courses 

12.8 

10.1 

2.8 

0.441 

Vocational  training 

7.9 

9.6 

-1.6 

0.626 

Participated  in  unpaid  work/subsidized  employment  (%) 

6.9 

10.7 

-3.8 

0.254 

Participated  in  an  education  activity  while  working  (%) 

14.9 

10.0 

4.9 

0.206 

Participated  in  an  employment  activity  while  working  (%) 

24.6 

15.1 

9.5  ** 

0.033 

Average  number  of  weeks  participating  in 

Job  search  activities 

10.1 

7.0 

3.1  ** 

0.033 

Education/training  activities 

6.1 

4.3 

1.8 

0.193 

Unpaid  work/subsidized  employment 

1.3 

1.4 

-0.2 

0.823 

Sample  size  (total  = 297) 

150 

147 

(continued) 
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Appendix  Table  A.8  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 


274 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.9 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Houston 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Standard 

Error 

Fo11ow-Ud  Year  1 

Ever  employed  (%) 

64.4 

63.8 

0.6 

0.758 

2.0 

Average  quarterly  employment  (%) 

42.6 

43.7 

-1.1 

0.476 

1.6 

Employed  every  quarter  (%) 

21.2 

23.2 

-2.1 

0.239 

1.8 

Employed  with  annual  earnings  over  $10,000  (%) 

12.4 

13.9 

-1.6 

0.268 

1.4 

Annual  earnings  ($) 

3,806 

3,936 

-130 

0.572 

230 

Ever  received  TANF  (%) 

85.7 

84.0 

1.7 

0.256 

1.5 

Ever  received  food  stamps  (%) 

91.5 

91.6 

-0.2 

0.878 

1.2 

Total  income3  ($) 

7,857 

7,888 

-31 

0.888 

220 

Fo11ow-Ud  Year  2 

Ever  employed  (%) 

60.0 

58.8 

1.3 

0.537 

2.0 

Average  quarterly  employment  (%) 

43.3 

41.7 

1.6 

0.359 

1.7 

Employed  every  quarter  (%) 

26.1 

24.5 

1.5 

0.411 

1.8 

Employed  with  annual  earnings  over  $10,000  (%) 

17.1 

17.5 

-0.4 

0.804 

1.6 

Annual  earnings  ($) 

4,576 

4,568 

7 

0.980 

282 

Ever  received  TANF  (%) 

56.0 

51.3 

4.8  ** 

0.022 

2.1 

Ever  received  food  stamps  (%) 

82.1 

80.2 

1.9 

0.232 

1.6 

Total  income3  ($) 

8,070 

8,011 

59 

0.827 

269 

Fo11ow-Ud  Year  3 

Ever  employed  (%) 

59.4 

60.3 

-1.0 

0.642 

2.0 

Average  quarterly  employment  (%) 

43.0 

43.3 

-0.2 

0.896 

1.7 

Employed  every  quarter  (%) 

26.6 

25.9 

0.8 

0.681 

1.9 

Employed  with  annual  earnings  over  $10,000  (%) 

17.6 

19.9 

-2.3 

0.159 

1.6 

Annual  earnings  ($) 

4,986 

5,006 

-20 

0.950 

314 

Ever  received  TANF  (%) 

42.7 

42.5 

0.2 

0.932 

2.1 

Ever  received  food  stamps  (%) 

79.6 

80.5 

-0.9 

0.599 

1.7 

Total  income3  ($) 

8,418 

8,417 

2 

0.995 

302 

Fo11ow-Ud  Year  4 

Ever  employed  (%) 

59.3 

59.9 

-0.6 

0.763 

2.0 

Average  quarterly  employment  (%) 

44.8 

44.7 

0.0 

0.981 

1.8 

Employed  every  quarter  (%) 

29.6 

29.6 

0.0 

0.990 

1.9 

Employed  with  annual  earnings  over  $10,000  (%) 

21.5 

22.4 

-1.0 

0.580 

1.7 

Annual  earnings  ($) 

5,519 

5,600 

-80 

0.809 

333 

Ever  received  TANF  (%) 

32.8 

30.6 

2.2 

0.273 

2.0 

Ever  received  food  stamps  (%) 

77.7 

77.1 

0.6 

0.736 

1.8 

Total  income3  ($) 

9,029 

8,893 

136 

0.673 

322 

Sample  size  (total  = 2,032) 

1,009 

1,023 

(continued) 
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Appendix  Table  A.9  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Total  income  does  not  include  the  value  of  the  stipend. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.  10 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

Houston 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Enrnlovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment"1 

1.4 

1.6 

-0.2 

0.140 

Average  quarterly  employment  (%) 

43.4 

43.3 

0.1 

0.963 

Percentage  of  quarters  employed  (%) 

Never  employed 

18.0 

16.9 

1.2 

0.461 

1-25 

23.0 

23.8 

-0.8 

0.653 

26-50 

19.2 

19.6 

-0.5 

0.798 

51-75 

16.7 

17.9 

-1.2 

0.454 

76-100 

23.1 

21.8 

1.4 

0.431 

Employed  entire  follow-up  period  (%) 

7.5 

8.5 

-1.0 

0.385 

Had  employment  spell  of  at  least  4 quarters  (%) 

54.9 

53.4 

1.6 

0.443 

Average  number  of  employers  during  follow-up  period 

3.2 

3.2 

0.0 

0.815 

Number  of  employers  (%) 

Never  employed 

18.0 

16.9 

1.2 

0.461 

1 to  2 

31.0 

33.2 

-2.2 

0.292 

3 to  4 

24.3 

24.1 

0.2 

0.920 

5 to  8 

20.6 

20.9 

-0.3 

0.861 

More  than  8 

6.2 

5.0 

1.2 

0.254 

Average  number  of  quarters  in  first  employment  spell 

4.2 

4.4 

-0.2 

0.349 

Number  of  quarters  until  first  employment  spell 

4.5 

4.5 

0.1 

0.817 

Average  number  of  employment  spells 

1.6 

1.6 

0.0 

0.696 

Average  length,  longest  employment  spell  (quarters) 

5.3 

5.4 

-0.1 

0.656 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

18.0 

16.9 

1.2 

0.461 

1 to  2 

19.8 

20.6 

-0.8 

0.637 

3 to  4 

16.5 

16.9 

-0.5 

0.776 

5 to  8 

23.2 

21.4 

1.8 

0.321 

9 to  12 

9.9 

10.8 

-0.9 

0.511 

13  to  16 

12.7 

13.5 

-0.8 

0.594 

(continued) 
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Appendix  Table  A.10  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Average  number  of  unemployment  spells 

1.8 

1.7 

0.1 

0.142 

Length  of  longest  unemployment  spell,  in  quarters 

7.5 

7.5 

0.0 

0.990 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

7.5 

8.5 

-1.0 

0.385 

1 to  2 

18.1 

15.7 

2.4 

0.134 

3 to  4 

14.3 

14.0 

0.3 

0.853 

5 to  8 

20.8 

23.1 

-2.4 

0.193 

9 to  12 

13.7 

13.5 

0.3 

0.869 

13  to  16 

25.6 

25.2 

0.4 

0.817 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

4,722 

4,777 

-56 

0.819 

Average  annual  earnings  (%) 

$0 

18.0 

16.9 

1.2 

0.461 

$l-$  1,999 

31.6 

31.2 

0.4 

0.857 

$2,000-84,999 

17.2 

19.6 

-2.5 

0.146 

$5,000-$9,999 

17.6 

15.6 

2.0 

0.215 

$10,000-814,999 

9.1 

8.7 

0.4 

0.777 

$15,000-819,999 

3.2 

4.9 

-1.7  * 

0.053 

$20,000  or  higher 

3.4 

3.1 

0.3 

0.720 

Average  annual  earnings  of  $10,000  or  more  (%) 

15.7 

16.7 

-1.0 

0.484 

Quarters  with  earnings  of  $3,500  or  more  (%) 

12.9 

13.8 

-0.9 

0.303 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

18.0 

16.9 

1.2 

0.461 

No  quarters  with  earnings  above  $3,500 

43.9 

43.6 

0.3 

0.897 

1 to  2 

12.7 

13.9 

-1.1 

0.451 

3 to  4 

7.0 

7.2 

-0.2 

0.848 

5 to  8 

9.8 

8.2 

1.6 

0.204 

9 to  12 

5.2 

5.9 

-0.6 

0.524 

13  to  16 

3.4 

4.4 

-1.0 

0.187 

Comparison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

22.5 

21.4 

1.0 

0.538 

Earnings  decreased  (%) 

No  longer  employed 

18.2 

18.7 

-0.4 

0.803 

Earnings  decreased  by  less  than  $250 

2.6 

1.4 

1.2  ** 

0.045 

Earnings  decreased  by  $250  or  more 

13.8 

15.2 

-1.4 

0.351 

Earnings  increased  (%) 

Became  employed 

13.2 

14.8 

-1.7 

0.282 

Earnings  increased  by  less  than  $250 

2.5 

2.2 

0.4 

0.602 

Earnings  increased  by  $250  or  more 

27.0 

26.1 

0.9 

0.631 

Average  earnings  per  quarter  employed  ($) 

2,719 

2,755 

-36 

Sample  size  (total  = 2,032) 

1,009 

1,023 

(continued) 
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Appendix  Table  A.10  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control 
groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only 
for  sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of 
program  group  and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not 
necessarily  be  attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously.  Zeroes  were  given  to  those 
who  were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.ll 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Houston 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

62.4 

64.2 

-1.8 

0.753 

Currently  employed 

40.9 

35.2 

5.7 

0.327 

No  longer  employed 

21.6 

29.0 

-7.4 

0.143 

Current  working  status  (%) 

Full  time 

32.6 

23.8 

OO 

oo 

* 

0.094 

Part  time 

8.2 

11.4 

-3.2 

0.370 

Currently  employed  at  a "good  job"a  (%) 

15.1 

9.8 

5.3 

0.174 

Hours 

Average  hours  per  week 

14.2 

11.7 

2.4 

0.245 

Total  hours  per  week  (%) 

Less  than  30 

8.2 

11.4 

-3.2 

0.370 

30-34 

6.3 

1.8 

4.5  * 

0.054 

35-44 

22.4 

19.3 

3.2 

0.513 

45  or  more 

4.0 

2.8 

1.2 

0.575 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

5.5 

4.6 

0.9 

0.732 

$5.00 -$6.99 

8.0 

10.9 

-3.0 

0.387 

$7.00  - $8.99 

17.9 

10.3 

7.6  * 

0.064 

$9.00  or  more 

9.5 

9.4 

0.1 

0.976 

Average  weekly  earnings  ($) 

104 

98 

6 

0.736 

Total  earnings  per  week  (%) 

Less  than  $200 

13.0 

11.9 

1.1 

0.783 

$201-$300 

14.7 

9.5 

5.2 

0.180 

$301-$500 

13.2 

10.4 

2.8 

0.470 

$500  or  more 

0.0 

3.4 

-3.4  ** 

0.025 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

12.8 

14.2 

-1.4 

0.729 

Paid  vacation 

16.0 

15.0 

1.0 

0.812 

Paid  holidays  other  than  Christmas  and  New  Year 

16.6 

10.9 

5.7 

0.161 

Dental  benefits 

13.4 

10.1 

3.3 

0.388 

A retirement  plan 

10.7 

8.8 

1.9 

0.594 

A health  plan  or  medical  insurance 

16.1 

12.2 

3.9 

0.342 

(continued) 
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Appendix  Table  A.  11  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Scheduleb(%) 

Regular 

24.6 

18.5 

6.1 

0.204 

Split  shift 

0.6 

0.1 

0.5 

0.455 

Irregular 

0.6 

2.1 

-1.6 

0.234 

Evening  shift 

3.2 

5.6 

-2.5 

0.311 

Night  shift 

2.0 

2.7 

-0.7 

0.713 

Rotating  shift 

7.7 

4.4 

3.4 

0.232 

Other  schedule 

2.0 

0.0 

2.0  * 

0.095 

Odd  job 

Job  skills 

0.2 

1.8 

-1.6 

0.155 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

26.7 

26.5 

0.2 

0.976 

Work  with  computers 

16.4 

13.9 

2.6 

0.544 

Arithmetic  skills 

22.7 

23.1 

-0.4 

0.939 

Customer  contact 

38.1 

32.5 

5.6 

0.327 

Sample  size  (total  = 297) 150 147 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job”  is  a job  in  which  a 
respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined  as 
one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to  nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.12 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Los  Angeles  EJC 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

42.1 

53.9 

-11.8 

***  0.004 

Average  number  of  contacts  with  staff 

7.3 

7.9 

-0.6 

0.701 

Talked  with  staff  in  past  4 weeks  (%) 

16.2 

20.1 

-3.9 

0.220 

Staff/case  manager  ever  talked  with  respondent's  employer  (%) 

7.2 

5.8 

1.4 

0.487 

Services 

Received  help  with  support  services  (%) 

46.4 

48.4 

-2.0 

0.629 

Received  help  with  basic  needs  (%) 

42.2 

45.0 

-2.8 

0.485 

Received  help  with  public  benefits  (%) 

59.8 

63.0 

-3.2 

0.429 

Received  help  with  job  preparation  (%) 

47.4 

46.0 

1.5 

0.717 

Received  help  with  retention/advancement  (%) 

21.9 

18.8 

3.1 

0.362 

Finding  a better  job  while  working 

4.6 

6.6 

-2.0 

0.303 

Other  activities  while  working 

5.6 

3.6 

2.0 

0.250 

Career  assessment 

15.9 

12.0 

4.0 

0.169 

Dealing  with  problems  on  the  job 

5.4 

5.1 

0.2 

0.899 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

6.9 

4.9 

2.0 

0.296 

ParticiDation 

Participated  in  any  activity  (%) 

77.3 

76.6 

0.7 

0.850 

Participated  in  any  employment-related  activity3  (%) 

71.4 

70.4 

1.1 

0.781 

Participated  in  a job  search  activity  (%) 

70.8 

70.0 

0.9 

0.818 

Group  job  search/job  club 

62.1 

61.2 

0.9 

0.824 

Individual  job  search 

48.1 

48.9 

-0.8 

0.848 

Participated  in  an  education/training  activity  (%) 

36.2 

41.5 

-5.3 

0.190 

ABE/GED  and  ESL 

17.1 

17.1 

0.0 

0.997 

College  courses 

13.5 

17.6 

-4.1 

0.158 

Vocational  training 

14.8 

13.8 

1.0 

0.743 

Participated  in  unpaid  work/subsidized  employment  (%) 

6.9 

5.9 

0.9 

0.650 

Participated  in  an  education  activity  while  working  (%) 

11.4 

12.9 

-1.5 

0.571 

Participated  in  an  employment  activity  while  working  (%) 

10.4 

11.0 

-0.6 

0.805 

Average  number  of  weeks  participating  in 

Job  search  activities 

5.4 

6.3 

-0.9 

0.215 

Education/training  activities 

7.4 

7.8 

-0.4 

0.734 

Unpaid  work/subsidized  employment 

0.9 

1.5 

-0.5 

0.308 

Sample  size  (total  = 608) 

311 

297 

(continued) 
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Appendix  Table  A.12  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discuseed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.13 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Los  Angeles  EJC 

ERA  Control  Difference  Standard 

Outcome  Group  Group  (Impact)  P-Value  Error 

Follow-Up  Year  1 


Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  2 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  3 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 


65.7 

65.7 

-0.1 

0.982 

2.6 

46.4 

43.9 

2.5 

0.248 

2.2 

28.1 

23.8 

4.3  * 

0.080 

2.4 

23.5 

20.3 

3.3 

0.159 

2.3 

5,429 

5,031 

397 

0.322 

401 

99.3 

99.2 

0.1 

0.815 

0.5 

93.6 

93.0 

0.6 

0.635 

1.3 

13,740 

13,316 

424 

0.258 

375 

64.8 

67.1 

-2.3 

0.383 

2.6 

49.7 

50.3 

-0.6 

0.809 

2.4 

34.9 

33.8 

1.1 

0.679 

2.7 

30.8 

33.9 

-3.1 

0.240 

2.6 

7,615 

7,845 

-230 

0.672 

542 

81.0 

80.5 

0.5 

0.831 

2.3 

73.4 

74.5 

-1.1 

0.650 

2.5 

13,504 

13,343 

160 

0.751 

505 

65.7 

66.1 

-0.4 

0.881 

2.7 

52.2 

51.5 

0.7 

0.760 

2.4 

37.4 

37.0 

0.3 

0.906 

2.7 

33.7 

36.2 

-2.5 

0.354 

2.7 

8,875 

9,098 

-223 

0.715 

610 

58.2 

57.3 

0.9 

0.757 

2.8 

55.2 

52.3 

2.9 

0.291 

2.8 

13,425 

13,467 

-42 

0.943 

583 

Sample  size  (total  = 1,183)  598  585 
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SOURCE:  MDRC  calculations  from  state  administrative  records. 


NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.14 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-3): 

Los  Angeles  EJC 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment3 

1.8 

1.4 

Q 4 ** 

0.011 

Average  quarterly  employment  (%) 

49.5 

48.6 

0.9 

0.640 

Percentage  of  quarters  employed  (%) 

Never  employed 

17.4 

17.2 

0.2 

0.925 

1-25 

17.7 

19.2 

-1.5 

0.516 

26-50 

18.0 

17.5 

0.5 

0.818 

51-75 

17.2 

19.7 

-2.5 

0.261 

76-100 

29.7 

26.4 

3.3 

0.188 

Employed  entire  follow-up  period  (%) 

16.4 

11.3 

5 l *** 

0.010 

Had  employment  spell  of  at  least  4 quarters  (%) 

56.3 

55.6 

0.7 

0.796 

Average  number  of  employers  during  follow-up  period 

2.5 

2.7 

-0.2 

0.175 

Number  of  employers  (%) 

Never  employed 

17.4 

17.2 

0.2 

0.925 

1 to  2 

40.8 

39.5 

1.3 

0.645 

3 to  4 

26.5 

25.4 

1.2 

0.645 

5 to  8 

14.1 

15.2 

-1.1 

0.601 

More  than  8 

1.1 

2.8 

-1.6  ** 

0.043 

Average  number  of  quarters  in  first  employment  spell 

4.7 

4.3 

0.3 

0.190 

Number  of  quarters  until  first  employment  spell 

3.4 

3.5 

-0.2 

0.461 

Average  number  of  employment  spells 

1.2 

1.3 

0.0 

0.495 

Average  length,  longest  employment  spell  (quarters) 

5.2 

5.0 

0.2 

0.351 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

17.4 

17.2 

0.2 

0.925 

1 to  2 

19.4 

19.1 

0.3 

0.909 

3 to  4 

13.6 

14.1 

-0.5 

0.787 

5 to  8 

22.0 

24.6 

-2.6 

0.285 

9 to  12 

27.6 

24.9 

2.7 

0.275 

(continued) 
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Appendix  Table  A.14  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Average  number  of  unemployment  spells 

1.3 

1.4 

-0.1  ** 

0.047 

Length  of  longest  unemployment  spell,  in  quarters 

5.3 

5.3 

0.0 

0.953 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

16.4 

11.3 

5 l *** 

0.010 

1 to  2 

18.9 

21.7 

-2.8 

0.221 

3 to  4 

14.0 

17.0 

-3.1 

0.149 

5 to  8 

24.1 

23.7 

0.4 

0.876 

9 to  12 

26.7 

26.3 

0.4 

0.867 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

7,306 

7,325 

-18 

0.967 

Average  annual  earnings  (%) 

$0 

17.4 

17.2 

0.2 

0.925 

$141,999 

22.1 

22.2 

0.0 

0.997 

$2,00044,999 

14.6 

12.5 

2.1 

0.291 

$5,00049,999 

17.2 

19.5 

-2.3 

0.311 

$10,000414,999 

10.5 

12.0 

-1.5 

0.411 

$15,000419,999 

8.8 

9.9 

-1.1 

0.504 

$20,000  or  higher 

9.4 

6.8 

2.6  * 

0.086 

Average  annual  earnings  of  $10,000  or  more  (%) 

28.6 

28.7 

0.0 

0.992 

Quarters  with  earnings  of  $3,500  or  more  (%) 

23.5 

23.9 

-0.4 

0.813 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

17.4 

17.2 

0.2 

0.925 

No  quarters  with  earnings  above  $3,500 

30.6 

28.1 

2.5 

0.338 

1 to  2 

13.3 

13.7 

-0.4 

0.854 

3 to  4 

10.5 

12.3 

-1.8 

0.328 

5 to  8 

15.5 

16.8 

-1.4 

0.522 

9 to  12 

12.7 

11.8 

0.8 

0.650 

Comparison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  3 

Not  employed  in  either  Year  1 or  Year  3 

20.2 

20.2 

0.1 

0.983 

Earnings  decreased  (%) 

No  longer  employed 

14.0 

13.7 

0.4 

0.862 

Earnings  decreased  by  less  than  $250 

2.2 

1.9 

0.3 

0.745 

Earnings  decreased  by  $250  or  more 

15.3 

11.7 

3.7  * 

0.065 

Earnings  increased  (%) 

Became  employed 

14.1 

14.1 

0.0 

0.995 

Earnings  increased  by  less  than  $250 

2.0 

1.9 

0.1 

0.927 

Earnings  increased  by  $250  or  more 

32.1 

35.8 

-3.7 

0.168 

Average  earnings  per  quarter  employed  ($) 

3,695 

3,773 

-79 

Sample  size  (total  = 1,183) 

598 

585 

(continued) 
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Appendix  Table  A.14  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups 
arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for 
sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group 
and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be 
attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously.  Zeroes  were  given  to  those  who 
were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.15 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Los  Angeles  EJC 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

56.4 

57.1 

-0.7 

0.868 

Currently  employed 

37.1 

36.3 

0.8 

0.838 

No  longer  employed 

19.4 

20.8 

-1.5 

0.659 

Current  working  status  (%) 

Full  time 

27.1 

27.2 

-0.1 

0.974 

Part  time 

10.0 

9.1 

0.9 

0.705 

Currently  employed  at  a "good  job"a  (%) 

15.3 

14.3 

1.0 

0.719 

Hours 

Average  hours  per  week 

12.8 

12.6 

0.2 

0.901 

Total  hours  per  week  (%) 

Less  than  30 

10.0 

9.1 

0.9 

0.711 

30-34 

3.3 

5.3 

-2.0 

0.224 

35-44 

20.3 

17.9 

2.4 

0.449 

45  or  more 

2.8 

4.2 

-1.5 

0.336 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.2 

1.4 

-0.2 

0.860 

$5.00 -$6.99 

5.3 

6.3 

-1.0 

0.604 

$7.00  - $8.99 

14.1 

11.8 

2.3 

0.401 

$9.00  or  more 

16.4 

16.8 

-0.4 

0.902 

Average  weekly  earnings  ($) 

121 

123 

-2 

0.889 

Total  earnings  per  week  (%) 

Less  than  $200 

8.2 

6.9 

1.3 

0.562 

$201-$300 

10.7 

9.7 

0.9 

0.711 

$301-$500 

14.4 

14.3 

0.1 

0.969 

$500  or  more 

3.8 

5.5 

-1.7 

0.343 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

12.9 

13.2 

-0.3 

0.908 

Paid  vacation 

16.2 

17.1 

-0.9 

0.759 

Paid  holidays  other  than  Christmas  and  New  Year 

16.2 

15.1 

1.1 

0.715 

Dental  benefits 

11.3 

12.4 

-1.1 

0.674 

A retirement  plan 

12.9 

12.5 

0.4 

0.875 

A health  plan  or  medical  insurance 

13.4 

15.3 

-2.0 

0.492 

(continued) 
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Appendix  Table  A.15  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Schedule11  (%) 

Regular 

20.2 

20.7 

-0.5 

0.884 

Split  shift 

1.2 

1.8 

-0.6 

0.545 

Irregular 

2.3 

4.3 

-2.0 

0.169 

Evening  shift 

3.7 

3.9 

-0.3 

0.857 

Night  shift 

2.3 

2.0 

0.3 

0.824 

Rotating  shift 

4.1 

3.1 

0.9 

0.545 

Other  schedule 

0.7 

0.0 

0.7 

0.159 

Odd  job 

Job  skills 

2.3 

0.7 

1.6 

0.107 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

25.1 

27.3 

-2.2 

0.547 

Work  with  computers 

19.0 

18.5 

0.5 

0.873 

Arithmetic  skills 

19.5 

18.3 

1.2 

0.716 

Customer  contact 

33.8 

30.3 

3.5 

0.362 

Sample  size  (total  = 608)  3 1 1 297 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "goodjob"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.16 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Salem 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

84.9 

79.0 

5.9 

0.197 

Average  number  of  contacts  with  staff 

22.0 

15.4 

6.6  ** 

0.042 

Talked  with  staff  in  past  4 weeks  (%) 

41.7 

41.0 

0.7 

0.907 

Staff  ever  talked  with  respondent's  employer  (%) 

12.8 

9.6 

3.2 

0.409 

Services 

Received  help  with  support  services  (%) 

55.0 

46.9 

8.1 

0.167 

Received  help  with  basic  needs  (%) 

58.3 

57.1 

1.2 

0.838 

Received  help  with  public  benefits  (%) 

83.1 

78.2 

4.8 

0.303 

Received  help  with  job  preparation  (%) 

51.0 

49.6 

1.4 

0.815 

Received  help  with  retention/advancement  (%) 

34.1 

20.4 

13.8  ** 

0.011 

Finding  a better  job  while  working 

9.2 

3.4 

59** 

0.036 

Other  activities  while  working 

20.8 

12.5 

8.3  * 

0.067 

Career  assessment 

16.7 

9.9 

6.8  * 

0.098 

Dealing  with  problems  on  the  job 

15.2 

4.7 

10  5 *** 

0.003 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

15.8 

8.1 

7.6  ** 

0.047 

ParticiDation 

Participated  in  any  activity  (%) 

86.5 

78.1 

8.4  * 

0.063 

Participated  in  any  employment-related  activity3  (%) 

78.1 

70.5 

7.7 

0.140 

Participated  in  a job  search  activity  (%) 

77.6 

70.3 

7.2 

0.166 

Group  job  search/job  club 

62.9 

49.6 

13.2  ** 

0.021 

Individual  job  search 

67.0 

61.0 

6.0 

0.292 

Participated  in  an  education/training  activity  (%) 

28.6 

32.8 

-4.1 

0.462 

ABE/GED  and  ESL 

10.8 

6.5 

4.3 

0.190 

College  courses 

15.8 

18.9 

-3.0 

0.494 

Vocational  training 

5.7 

11.0 

-5.3 

0.110 

Participated  in  unpaid  work/ subsidized  employment  (%) 

15.0 

13.7 

1.2 

0.758 

Participated  in  an  education  activity  while  working  (%) 

12.0 

14.0 

-2.0 

0.618 

Participated  in  an  employment  activity  while  working  (%) 

23.2 

12.7 

10.5  ** 

0.023 

Average  number  of  weeks  participating  in 

Job  search  activities 

14.9 

8.9 

6.0  *** 

0.001 

Education/training  activities 

5.0 

5.5 

-0.5 

0.745 

Unpaid  work/subsidized  employment 

3.4 

2.6 

0.8 

0.490 

Sample  size  (total  = 300) 

152 

148 

(continued) 
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Appendix  Table  A.16  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.17 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Salem 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

Impact 

P-Value 

Standard 

Error 

Fo11ow-Ud  Year  1 

Ever  employed  (%) 

62.6 

62.8 

-0.2 

0.939 

2.4 

Average  quarterly  employment  (%) 

39.9 

42.3 

-2.4 

0.211 

1.9 

Employed  every  quarter  (%) 

19.2 

22.0 

-2.8 

0.164 

2.0 

Employed  with  annual  earnings  over  $10,000  (%) 

15.0 

16.3 

-1.3 

0.483 

1.8 

Annual  earnings  ($) 

3,840 

4,095 

-254 

0.376 

287 

Ever  received  TANF  (%) 

64.4 

57.7 

^ 7 *** 

0.007 

2.5 

Ever  received  food  stamps  (%) 

93.5 

92.3 

1.2 

0.339 

1.3 

Total  income  ($) 

8,813 

8,564 

249 

0.397 

294 

Fo11ow-Ud  Year  2 

Ever  employed  (%) 

62.0 

61.1 

0.9 

0.715 

2.4 

Average  quarterly  employment  (%) 

42.8 

43.9 

-1.1 

0.597 

2.0 

Employed  every  quarter  (%) 

25.4 

27.6 

-2.2 

0.320 

2.2 

Employed  with  annual  earnings  over  $10,000  (%) 

22.7 

24.3 

-1.6 

0.448 

2.1 

Annual  earnings  ($) 

5,342 

5,588 

-247 

0.522 

385 

Ever  received  TANF  (%) 

42.0 

35.6 

g 5 *** 

0.008 

2.4 

Ever  received  food  stamps  (%) 

78.4 

78.3 

0.1 

0.958 

2.0 

Total  income  ($) 

9,160 

8,900 

260 

0.499 

385 

FoIIow-Ud  Year  3 

Ever  employed  (%) 

58.6 

61.7 

-3.1 

0.211 

2.5 

Average  quarterly  employment  (%) 

44.6 

48.4 

-3.7  * 

0.082 

2.2 

Employed  every  quarter  (%) 

30.9 

34.2 

-3.4 

0.151 

2.3 

Employed  with  annual  earnings  over  $10,000  (%) 

27.6 

29.4 

-1.7 

0.432 

2.2 

Annual  earnings  ($) 

6,593 

6,873 

-280 

0.528 

444 

Ever  received  TANF  (%) 

29.8 

26.6 

3.1 

0.167 

2.3 

Ever  received  food  stamps  (%) 

69.6 

70.7 

-1.1 

0.622 

2.3 

Total  income  ($) 

9,524 

9,642 

-118 

0.789 

439 

Sample  size  (total  = 1 ,504) 

742 

762 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  meaures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.18 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-3): 

Salem 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment3 

1.3 

1.4 

-0.1 

0.385 

Average  quarterly  employment  (%) 

42.4 

44.9 

-2.4 

0.153 

Percentage  of  quarters  employed  (%) 

Never  employed 

18.7 

19.6 

-0.8 

0.666 

1-25 

23.9 

21.8 

2.2 

0.323 

26-50 

19.0 

17.2 

1.8 

0.364 

51-75 

18.2 

17.7 

0.5 

0.787 

76-100 

20.2 

23.8 

-3.7  * 

0.079 

Employed  entire  follow-up  period  (%) 

10.1 

12.9 

-2.8  * 

0.086 

Had  employment  spell  of  at  least  4 quarters  (%) 

47.5 

49.2 

-1.7 

0.494 

Average  number  of  employers  during  follow-up  period 

2.5 

2.6 

-0.1 

0.333 

Number  of  employers  (%) 

Never  employed 

18.7 

19.6 

-0.8 

0.666 

1 to  2 

40.3 

37.7 

2.7 

0.297 

3 to  4 

24.7 

25.0 

-0.3 

0.879 

5 to  8 

13.8 

15.1 

-1.3 

0.469 

More  than  8 

2.5 

2.7 

-0.2 

0.834 

Average  number  of  quarters  in  first  employment  spell 

3.7 

4.0 

I 

o 

4^ 

* 

0.068 

Number  of  quarters  until  first  employment  spell 

3.7 

3.7 

0.0 

0.969 

Average  number  of  employment  spells 

1.3 

1.2 

0.0 

0.349 

Average  length,  longest  employment  spell  (quarters) 

4.3 

4.6 

-0.3 

0.110 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

18.7 

19.6 

-0.8 

0.666 

1 to  2 

25.8 

22.5 

3.3 

0.132 

3 to  4 

14.8 

15.6 

-0.8 

0.660 

5 to  8 

22.8 

20.6 

2.1 

0.317 

9 to  12 

17.9 

21.7 

-3.8  * 

0.058 

(continued) 


294 


Appendix  Table  A.18  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Average  number  of  unemployment  spells 

1.4 

1.4 

0.1  ** 

0.038 

Length  of  longest  unemployment  spell,  in  quarters 

5.9 

5.8 

0.2 

0.370 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

10.1 

12.9 

-2.8  * 

0.086 

1 to  2 

16.9 

16.0 

0.9 

0.632 

3 to  4 

18.1 

16.9 

1.3 

0.525 

5 to  8 

23.3 

23.8 

-0.4 

0.842 

9 to  12 

31.6 

30.5 

1.1 

0.645 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

5,258 

5,519 

-260 

0.425 

Average  annual  earnings  (%) 

SO 

18.7 

19.6 

-0.8 

0.666 

$141,999 

28.5 

26.0 

2.5 

0.279 

$2,00044,999 

15.9 

15.6 

0.3 

0.877 

$5,00049,999 

17.3 

17.1 

0.2 

0.928 

$10,000414,999 

9.4 

10.5 

-1.1 

0.492 

$15,000419,999 

5.7 

6.8 

-1.1 

0.375 

$20,000  or  higher 

4.4 

4.4 

0.1 

0.956 

Average  annual  earnings  of  $10,000  or  more  (%) 

19.5 

21.7 

-2.1 

0.293 

Quarters  with  earnings  of  $3,500  or  more  (%) 

16.7 

18.2 

-1.6 

0.234 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

18.7 

19.6 

-0.8 

0.666 

No  quarters  with  earnings  above  $3,500 

37.2 

36.3 

0.9 

0.720 

1 to  2 

16.6 

13.4 

3.2  * 

0.083 

3 to  4 

9.0 

10.0 

-1.0 

0.516 

5 to  8 

10.5 

12.7 

-2.2 

0.179 

9 to  12 

8.0 

8.0 

-0.1 

0.967 

ComDarison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  3 

Not  employed  in  either  Year  1 or  Year  3 

22.5 

22.6 

-0.2 

0.931 

Earnings  decreased  (%) 

No  longer  employed 

18.9 

15.7 

3.2 

0.101 

Earnings  decreased  by  less  than  $250 

1.9 

2.4 

-0.6 

0.466 

Earnings  decreased  by  $250  or  more 

11.0 

11.4 

-0.4 

0.786 

Earnings  increased  (%) 

Became  employed 

14.9 

14.6 

0.4 

0.844 

Earnings  increased  by  less  than  $250 

1.2 

1.6 

-0.5 

0.429 

Earnings  increased  by  $250  or  more 

29.6 

31.6 

-2.0 

0.395 

Average  earnings  per  quarter  employed  ($) 

3,098 

3,077 

21 

Sample  size  (total  = 1 ,504) 

742 

762 

(continued) 
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Appendix  Table  A.  18  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups 
arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for 
sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program 
group  and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be 
attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously.  Zeroes  were  given  to  those  who 
were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  A.  19 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Salem 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

EmDlovment  status 

Ever  employed  since  random  assignment  (%) 

74.8 

75.9 

-1.1 

0.835 

Currently  employed 

40.4 

44.3 

-3.9 

0.489 

No  longer  employed 

34.4 

31.5 

2.9 

0.608 

Current  working  status  (%) 

Full  time 

30.2 

33.2 

-2.9 

0.588 

Part  time 

10.2 

11.2 

-1.0 

0.781 

Currently  employed  at  a "good  job"a  (%) 

14.3 

15.8 

-1.5 

0.720 

Hours 

Average  hours  per  week 

13.6 

14.6 

-1.0 

0.636 

Total  hours  per  week  (%) 

Less  than  30 

10.2 

11.2 

-1.0 

0.781 

30-34 

8.0 

10.0 

-1.9 

0.571 

35-44 

16.3 

19.0 

-2.7 

0.550 

45  or  more 

5.8 

4.1 

1.7 

0.514 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

2.7 

3.3 

-0.6 

0.757 

$5.00 -$6.99 

3.1 

4.3 

-1.2 

0.607 

$7.00  - $8.99 

18.9 

18.4 

0.5 

0.922 

$9.00  or  more 

15.7 

18.3 

-2.6 

0.551 

Average  weekly  earnings  ($) 

120 

132 

-11 

0.599 

Total  earnings  per  week  (%) 

Less  than  $200 

9.1 

12.3 

-3.2 

0.390 

$201-$300 

14.6 

12.0 

2.6 

0.532 

$301-$500 

11.5 

17.2 

-5.7 

0.164 

$500  or  more 

5.1 

2.8 

2.3 

0.311 

Benefits 

Currently  employed  and  receiving  employer 
provided  benefits  at  current  job  (%) 

Sick  days  with  full  pay 

11.6 

17.8 

-6.2 

0.134 

Paid  vacation 

16.1 

21.3 

-5.2 

0.256 

Paid  holidays  other  than  Christmas  and  New  Years 

12.9 

17.2 

-4.3 

0.312 

Dental  benefits 

14.0 

16.7 

-2.7 

0.525 

A retirement  plan 

11.3 

14.7 

-3.4 

0.397 

A health  plan  or  medical  insurance 

15.2 

20.9 

-5.6 

0.212 

(continued) 
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Appendix  Table  A.19  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Schedule11  (%) 

Regular 

22.9 

22.4 

0.5 

0.924 

Split  shift 

0.5 

3.5 

-3.0  * 

0.077 

Irregular 

3.5 

1.8 

1.6 

0.381 

Evening  shift 

4.3 

5.7 

-1.4 

0.601 

Night  shift 

3.9 

2.8 

1.1 

0.613 

Rotating  shift 

3.3 

6.7 

-3.4 

0.185 

Other  schedule 

0.0 

0.7 

-0.7 

0.310 

Odd  job 

J ob  skills 

2.0 

0.7 

1.3 

0.323 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

31.1 

38.3 

-7.2 

0.187 

Work  with  computers 

17.2 

17.5 

-0.2 

0.956 

Arithmetic  skills 

24.7 

30.7 

-6.0 

0.225 

Customer  contact 

34.9 

38.5 

-3.6 

0.512 

Sample  size  (total  = 300)  152  148 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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Appendix  B 

Supplementary  Exhibits  for  Chapter  4: 
Programs  Serving  Employed  TANF  Recipients 


Appendix  B presents  supplementary  boxes  and  tables  relating  to  the  Chicago,  Los  Angeles  RFS, 

and  Riverside  Phase  2 ERA  tests. 

Box 

B.  1 Program  Summary:  Chicago 

B.2  Program  Summary:  Los  Angeles  RFS 

B.3  Program  Summary:  Riverside  Phase  2:  Training  Focused  Group  and  Work  Plus  Group 

Table 

B.  1 Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Chicago 

B.2  Impacts  on  U1 -Covered  Employment,  Public  Assistance,  and  Measured  Income:  Chicago 

B.3  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4):  Chicago 

B.4  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Chicago 

B.5  Impacts  on  Employment  Retention,  Advancement,  and  TANF  Receipt  in  the 
Cumulative  Follow-Up  Period  (Years  1-4),  by  Study  Cohort:  Chicago 

B.6  Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42 -Month  Survey 
for  the  Early  Cohort:  Chicago 

B.7  Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42-Month  Survey 
for  the  Late  Cohort:  Chicago 
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Table 

B.8 

B.9 

B.10 

B.l  1 
B.12 

B.13 

B.14 

B.15 

B.16 

B.17 


Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Los  Angeles  RFS 

Impacts  on  UI -Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Los  Angeles  RFS 

Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-3):  Los  Angeles  RFS 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Los  Angeles  RFS 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Riverside  Phase  2 

Impacts  on  UI -Covered  Employment,  Public  Assistance,  and  Measured  Income: 
Riverside  Phase  2 

Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4):  Riverside  Phase  2 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Riverside  Phase  2 

Impacts  on  Services,  Participation,  and  Receipt  of  Education  Credentials  at  the 
12-Month  Survey:  Riverside  Phase  2 

Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4),  by  Educational  Attainment:  Riverside  Phase  2 
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Appendix  Box  B.l 


Program  Summary: 

Chicago 

“Mandatory,  multifaceted  services  provided  through  an  employment  intermediary” 
Goal:  Advance  employed  TANF  recipients  into  higher-paying  jobs 
Locations:  Selected  welfare  offices  in  Cook  County  (Chicago  area) 

Target  population:  TANF  recipients  who  worked  at  least  30  hours  per  week  for  at  least  six 
consecutive  months 

Implementation  schedule:  Random  assignment  occurred  from  February  2002  to  June  2003, 
and  the  program  operated  from  the  beginning  of  random  assignment  to  June  2004. 

Management  structure:  Operated  by  a for-profit  company,  Employment  and  Employer 
Services  (E&ES),  under  contract  to  DHS;  included  staff  performance  incentives 

Participation  requirements:  Besides  standard  participation  requirements  to  continue  to  receive 
the  TANF  grant,  ERA  added  the  additional  requirement  of  regular  contact  with  program  staff. 

Outreach  and  marketing:  Aggressive  marketing  and  outreach  strategy  with  tailored  messaging 
to  participants  and  financial  participation  incentives 

Staff-client  engagement: 

• Generic  staff  (not  team-based)  provided  intense  monitoring  and  follow-up. 

• Services  were  tailored  to  individual  participants,  with  little  flexibility  in  meeting  times 
and  locations. 

Job  preparation  and  placement  services:  Though  not  a priority  of  the  model,  staff  provided 
reemployment  services  because  a substantial  number  of  program  group  members  were  no 
longer  employed  when  they  first  had  contact  with  program  staff. 

Retention  services:  Though  not  a priority  of  the  model’s  design,  staff  connected  participants 
with  social  services  and  helped  provide  such  work  supports  as  child  care  subsidies. 

Advancement  services: 

• Job  search  activities  — Staff  identified  job  openings  with  comparable  or  greater  pay 
and  helped  with  resume  writing  and  scheduling  of  and  preparation  for  interviews. 

• Career  counseling  — An  employment  plan  was  developed  and  used  to  identify  short- 
and  long-term  goals,  barriers  to  achieving  them,  and  resources  for  overcoming  them. 

• Education  and  training  — Though  not  a priority  of  the  model,  discussions  about  and 
referrals  to  these  services  increased  over  the  study  period. 

Employer  linkages:  Staff  had  strong  preexisting  relationships  with  local  employers  and  were 
required  to  identify  jobs  within  those  firms  as  part  of  their  ERA  responsibilities. 

Key  funding  source:  State  TANF  program,  with  special  U.S.  Department  of  Labor  grant  after 
unspent  TANF  funds  were  rescinded,  resulting  in  a brief  funding  gap. 

(continued) 
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Appendix  Box  B.l  (continued) 


Implementation  challenges: 

• Difficulty  engaging  participants  and  maintaining  their  ongoing  participation 

• Unexpected  retention  and  reemployment  needs,  which  hampered  advancement  services 

• Ever-increasing  caseloads  (Service  eligibility  was  open-ended  and  distinct  from  TANF 
receipt.) 
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Appendix  Box  B.2 


Program  Summary: 

Los  Angeles  RFS  (Reach  For  Success) 

“Highly  individualized  and  flexible” 

Goal:  Help  working  TANF  recipients  retain  their  employment  and  secure  better  jobs 

Locations:  Region  1 (western  Los  Angeles  County  and  neighborhoods  surrounding  LA 
airport),  Region  5 (including  South  Central  Los  Angeles),  and  Region  6 (including  East  Los 
Angeles)  of  the  Los  Angeles  County  Greater  Avenues  for  Independence  (GAIN)  program 

Target  population:  Single-parent  welfare  recipients  who  were  GAIN  participants  and  who  had 
been  working  in  a full-time  job  of  32  hours  for  generally  more  than  30  days 

Implementation  schedule:  Random  assignment  occurred  from  July  2002  to  June  2004,  and  the 
program  operated  from  the  beginning  of  random  assignment  to  June  2005. 

Management  structure:  Operated  by  the  Welfare-to-Work  Division  of  the  Los  Angeles 
County  Department  of  Public  Social  Services 

Participation  requirements:  ERA-specific  activities  did  not  introduce  any  additional 
mandatory  TANF  participation  requirements. 

Outreach  and  marketing:  Staff  conducted  multifaceted  outreach  and  marketing  strategies  that 
included  the  development  of  marketing  materials,  offers  of  off-site  meetings  with  clients,  and 
emphasis  on  the  voluntary  and  individualized  nature  of  services. 

Staff/Client  Engagement: 

• Generic  staff  (not  team-based)  provided  intense  monitoring  and  follow-up. 

• Services  were  tailored  to  individual  participants,  with  considerable  flexibility  in 
meeting  times  and  locations. 

Job  preparation  and  placement  services:  Not  a priority  of  the  model  and  not  pursued  by  staff 

Retention  services: 

• Due  to  high  rates  of  job  loss,  staff  provided  considerable  reemployment  services,  which 
focused  on  securing  better  jobs  (wages,  benefits,  career  prospects)  rather  than  any  job. 

• Staff  facilitated  the  securing  of  work  supports  and  provided  social  service  referrals. 

Advancement  services: 

• Career  counseling  — Staff  developed  and  maintained  employment  plans  and  identified 
short-  and  long-term  goals,  barriers  to  achieving  those  goals,  and  steps  and  services  for 
overcoming  them. 

• Education  and  training  — Staff  encouraged  and  directed  clients  to  education  and 
training  providers  within  the  context  of  their  employment  plan. 

Employer  linkages:  Not  part  of  the  model  and  not  pursued  by  staff 

(continued) 
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Appendix  Box  B.2  (continued) 


Key  funding  source:  TANF  grant  money 

Implementation  challenges: 

• Difficulty  engaging  clients  and  maintaining  their  ongoing  participation 

• High  demand  for  reemployment  services,  which  often  precluded  delivery  of 
advancement  services 

• Limited  career  counseling  skills  among  staff 
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Appendix  Box  B.3 


Program  Summary: 

Riverside  Phase  2:  Training  Focused  Group  and  Work  Plus  Group 

“Balancing  work  and  training  for  advancement” 

Goal:  Employment  retention  and  advancement  among  employed  TANF  recipients 
Locations:  Riverside  County,  California 

Target  population:  TANF  recipients  who  had  worked  at  least  20  hours  per  week  for  at  least  30 
days 

Implementation  schedule:  Random  assignment  occurred  from  January  2001  to  October  2003. 
The  Training  Focused  program  started  in  September  2000  and  continued  through  October  2006. 
The  Work  Plus  program  started  in  January  1998  and  is  still  in  operation. 

Management  structure:  The  Training  Focused  program  was  operated  by  the  Welfare-to-Work 
Division  of  the  Economic  Development  Agency  (EDA).  The  Work  Plus  program  was  operated 
by  the  county  welfare  agency. 

Participation  requirements:  Riverside  “Phase  2”  was  subject  to  standard  TANF  rules  that 
required  32  hours  per  week  of  work.  Work  Plus  required  at  least  20  hours  of  work  per  week  and 
the  remaining  12  additional  hours  could  be  fulfilled  by  education  and  training  participation 
and/or  job  search  activities.  Training  Focused  had  no  specific  mix  of  activities  that  were  to 
fulfill  the  weekly  work  requirement;  in  other  words,  all  32  weekly  required  hours  could  be 
fulfilled  by  hours  of  education  and  training  participation,  at  least  temporarily. 

Outreach  and  marketing:  Staff  conducted  intensive  outreach,  including  recruitment  pitches 
customized  to  participants’  career  and  family  characteristics. 

Staff-client  engagement: 

• Generic  staff  (not  team-based  structure) 

• Services  tailored  to  individual  participants 

• Client  assessment  leading  to  development  of  employment  plans 

• Intense  ongoing  contact,  with  flexibility  in  their  means,  timing,  and  location 

Job  preparation  and  placement  services:  Not  a priority  of  the  model  and  not  pursued  by  staff. 

Retention  services:  Other  than  facilitating  work  supports  and  social  service  referrals,  not  a 
priority  of  the  model  and  not  pursued  by  staff 

Advancement  services:  Education  and  training  was  the  focus  of  both  models.  Work  Plus  staff 
recommended  a number  of  providers  from  which  participants  could  choose,  while  Training 
Focused  staff  directed  participants  to  particular  providers  and  programs.  Work  Plus  prioritized 
addressing  basic  education  needs  before  vocational  training,  while  Training  Focused  supported 
addressing  basic  education  needs  in  the  context  of  particular  skill-training  programs. 


(continued) 


305 


Appendix  Box  B.3  (continued) 


Employer  linkages:  Not  included  in  the  model  and  not  pursued  by  staff 

Key  funding  source:  Both  programs  were  primarily  funded  with  TANF  grant  money.  A key 
resource  for  providing  education  and  training  services  to  the  Training  Focused  program  was 
ACCESS,  a welfare-to-work  grant  won  by  EDA  from  the  U.S.  Department  of  Labor. 

Implementation  challenges: 

• Difficulty  engaging  participants  and  maintaining  their  ongoing  participation 

• Unexpected  retention  and  reemployment  needs  among  participants 

• Fluctuations  in  the  education  and  training  funding  streams 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.l 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Contacts 

Ever  had  contact  with  staff/ employment  program  (%) 

61.0 

31.3 

29  g *** 

0.000 

Average  number  of  contacts  with  staff 

9.8 

2.8 

q 9 *** 

0.000 

Talked  with  staff  in  past  4 weeks  (%) 

25.9 

9.6 

16.3  *** 

0.000 

Staff  ever  talked  with  respondent's  employer  (%) 

12.8 

4.3 

8 5 *** 

0.000 

Services 

Received  help  with  support  services  (%) 

39.6 

35.9 

3.7 

0.347 

Received  help  with  basic  needs  (%) 

32.1 

34.2 

-2.1 

0.591 

Received  help  with  public  benefits  (%) 

48.0 

54.8 

-6.8  * 

0.099 

Received  help  with  job  preparation  (%) 

38.2 

17.9 

20  3 *** 

0.000 

Received  any  help  with  retention/advancement  (%) 

37.0 

12.6 

24  5 *** 

0.000 

Finding  a better  job  while  working 

28.0 

5.2 

22.8  *** 

0.000 

Other  activities  while  working 

13.9 

2.3 

11.6  *** 

0.000 

Career  assessment 

21.7 

6.7 

1 *** 

0.000 

Dealing  with  problems  on  the  job 

8.7 

4.5 

4.2  ** 

0.043 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

7.1 

1.8 

5 4 *** 

0.002 

Participation 

Participated  in  any  activity  (%) 

64.0 

47.6 

16,4  *** 

0.000 

Participated  in  any  employment-related  activity3  (%) 

57.1 

37.8 

19  4 *** 

0.000 

Participated  in  a job  search  activity  (%) 

56.5 

35.7 

9Q  y *** 

0.000 

Group  job  search/job  club 

43.7 

18.6 

25  1 *** 

0.000 

Individual  job  search 

43.4 

28.5 

14  9 *** 

0.000 

Participated  in  an  education/training  activity  (%) 

23.3 

24.9 

-1.6 

0.648 

ABE/GED  and  ESL 

13.9 

13.8 

0.1 

0.972 

College  courses 

6.4 

6.6 

-0.2 

0.922 

Vocational  training 

4.5 

8.0 

-3.5  * 

0.075 

Participated  in  unpaid  work/ subsidized  employment  (%) 

7.0 

7.4 

-0.4 

0.862 

Participated  in  an  education  activity  while  working  (%) 

15.4 

15.0 

0.4 

0.904 

Participated  in  an  employment  activity  while  working  (%) 

22.6 

11.9 

IQ  7 *** 

0.001 

Average  number  of  weeks  participating  in 

Job  search  activities 

7.2 

4.9 

9 3 ** 

0.029 

Education/ training  activities 

2.9 

3.3 

-0.5 

0.510 

Unpaid  work/subsidized  employment 

1.2 

1.1 

0.1 

0.807 

Sample  size  (total  = 598) 

306 

292 

(continued) 
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Appendix  Table  B.l  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES: The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.2 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Chicago 

ERA  Control  Difference  Standard 

Outcome  Group  Group  (Impact)  P-Value  Error 

Follow-Up  Year  1 


Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  in  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  2 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  in  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  3 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  in  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  4 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  in  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 


69.7 

66.2 

3.5  ** 

0.031 

1.6 

57.7 

55.3 

2.4  * 

0.087 

1.4 

45.2 

43.1 

2.1 

0.246 

1.8 

30.1 

27.0 

3.1  * 

0.084 

1.8 

6,221 

6,008 

214 

0.354 

231 

85.1 

88.1 

-3.0  * 

0.067 

1.6 

99.5 

98.8 

0.8  * 

0.073 

0.4 

11,625 

11,650 

-26 

0.909 

224 

65.4 

60.8 

4.6  ** 

0.018 

2.0 

55.0 

50.1 

4 9 *** 

0.005 

1.8 

43.4 

38.9 

4.5  ** 

0.027 

2.0 

30.1 

28.5 

1.6 

0.412 

2.0 

6,564 

5,973 

591  ** 

0.048 

298 

31.9 

46.3 

-14  5 *** 

0.000 

2.3 

92.0 

90.8 

1.1 

0.393 

1.3 

11,329 

10,793 

536  * 

0.070 

295 

61.8 

59.4 

2.5 

0.229 

2.1 

51.6 

50.2 

1.3 

0.480 

1.9 

41.5 

39.9 

1.7 

0.434 

2.1 

33.2 

31.1 

2.1 

0.300 

2.1 

6,949 

6,467 

481 

0.158 

341 

20.5 

28.2 

_1  y *** 

0.000 

2.1 

86.0 

84.0 

2.0 

0.241 

1.7 

11,448 

10,911 

537 

0.119 

344 

65.4 

64.3 

1.2 

0.576 

2.1 

55.5 

53.8 

1.7 

0.373 

1.9 

44.6 

43.1 

1.5 

0.504 

2.2 

37.9 

34.4 

3.4 

0.106 

2.1 

8,135 

7,511 

624 

0.108 

387 

Sample  size  (total  = 1,728)  854  874 

(continued) 
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Appendix  Table  B.2  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 


310 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.3 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment11 

4.5 

4.4 

0.2 

0.417 

Average  quarterly  employment  (%) 

55.0 

52.4 

2.6  * 

0.069 

Percentage  of  quarters  employed  (%) 

Never  employed 

19.5 

20.8 

-1.3 

0.434 

1-25 

13.8 

15.2 

-1.4 

0.391 

26-50 

13.0 

12.8 

0.2 

0.902 

51-75 

12.8 

13.1 

-0.3 

0.868 

76-100 

40.9 

38.1 

2.8 

0.159 

Employed  entire  follow-up  period  (%) 

25.7 

23.0 

2.7 

0.137 

Had  employment  spell  of  at  least  4 quarters  (%) 

64.8 

62.9 

1.8 

0.342 

Average  number  of  employers  during  follow-up  period 

2.1 

2.0 

0.1 

0.329 

Number  of  employers  (%) 

Never  employed 

19.5 

20.8 

-1.3 

0.434 

1 to  2 

51.6 

52.0 

-0.4 

0.867 

3 to  4 

17.7 

18.6 

-0.9 

0.614 

5 to  8 

9.7 

7.1 

2.6  * 

0.051 

More  than  8 

1.6 

1.6 

0.0 

0.994 

Average  number  of  quarters  in  first  employment  spell 

7.0 

6.6 

0.4 

0.149 

Number  of  quarters  until  first  employment  spell 

4.1 

4.5 

-0.4  * 

0.072 

Average  number  of  employment  spells 

1.3 

1.2 

0.0 

0.457 

Average  length,  longest  employment  spell  (quarters) 

7.7 

7.3 

0.4  * 

0.091 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

19.5 

20.8 

-1.3 

0.434 

1 to  2 

10.6 

11.6 

-1.0 

0.486 

3 to  4 

9.9 

10.3 

-0.4 

0.766 

5 to  8 

17.5 

18.2 

-0.7 

0.696 

9 to  12 

11.4 

10.0 

1.4 

0.344 

13  to  16 

31.2 

29.1 

2.1 

0.299 

(continued) 


311 


Appendix  Table  B.3  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Average  number  of  unemployment  spells 

1.1 

1.1 

0.0 

0.836 

Length  of  longest  unemployment  spell,  in  quarters 

6.5 

6.9 

-0.4  * 

0.054 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

25.7 

23.0 

2.7 

0.137 

1 to  2 

13.4 

13.5 

-0.2 

0.922 

3 to  4 

12.3 

11.0 

1.4 

0.368 

5 to  8 

12.8 

14.6 

-1.7 

0.293 

9 to  12 

10.6 

9.9 

0.7 

0.632 

13  to  16 

25.2 

28.1 

-2.9 

0.107 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

6,967 

6,490 

477  * 

0.074 

Average  annual  earnings  (%) 

$0 

19.5 

20.8 

-1.3 

0.434 

$l-$  1,999 

15.8 

16.5 

-0.6 

0.727 

$2,000-$4,999 

13.3 

14.1 

-0.8 

0.620 

$5,000-$9,999 

20.2 

19.4 

0.8 

0.666 

$10,000-814,999 

17.6 

18.0 

-0.4 

0.809 

$15,000-$  19,999 

9.6 

7.9 

1.8 

0.181 

$20,000  or  higher 

4.0 

3.5 

0.5 

0.542 

Average  annual  earnings  of  $ 10,000  or  more  (%) 

31.2 

29.3 

1.9 

0.340 

Quarters  with  earnings  of  $3,500  or  more  (%) 

21.6 

19.0 

2.6  ** 

0.035 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

19.5 

20.8 

-1.3 

0.434 

No  quarters  with  earnings  above  $3,500 

29.7 

32.5 

-2.8 

0.195 

1 to  2 

12.9 

12.3 

0.6 

0.694 

3 to  4 

7.3 

7.7 

-0.4 

0.756 

5 to  8 

12.6 

11.2 

1.4 

0.370 

9 to  12 

9.3 

7.8 

1.5 

0.248 

13  to  16 

8.6 

7.7 

0.9 

0.464 

Comnarison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

22.6 

22.9 

-0.4 

0.814 

Earnings  decreased  (%) 

No  longer  employed 

12.1 

12.8 

-0.8 

0.630 

Earnings  decreased  by  less  than  $250 

3.0 

3.5 

-0.5 

0.596 

Earnings  decreased  by  $250  or  more 

13.9 

12.8 

1.1 

0.485 

Earnings  increased  (%) 

Became  employed 

7.8 

10.8 

-3.1  ** 

0.021 

Earnings  increased  by  less  than  $250 

3.4 

3.6 

-0.2 

0.797 

Earnings  increased  by  $250  or  more 

37.2 

33.6 

3.5  * 

0.095 

Average  earnings  per  quarter  employed  ($) 

3,170 

3,099 

71 

Sample  size  (total  = 1,728) 

854 

874 

(continued) 
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Appendix  Table  B.3  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control 
groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only 
for  sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of 
program  group  and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not 
necessarily  be  attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took 
place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after 
random  assignment,  even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given 
to  those  who  were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.4 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

82.9 

84.7 

-1.9 

0.536 

Currently  employed 

67.8 

67.6 

0.2 

0.956 

No  longer  employed 

15.0 

16.8 

-1.8 

0.559 

Current  working  status  (%) 

Full  time 

54.2 

55.9 

-1.7 

0.682 

Part  time 

13.6 

11.8 

1.9 

0.496 

Currently  employed  at  a "good  job"3  (%) 

13.8 

9.8 

4.0 

0.121 

Hours 

Average  hours  per  week 

23.5 

23.7 

-0.2 

0.910 

Total  hours  per  week  (%) 

Less  than  30 

13.6 

11.8 

1.9 

0.496 

30-34 

12.8 

14.0 

-1.2 

0.673 

35-44 

36.3 

36.3 

0.0 

0.998 

45  or  more 

5.1 

4.6 

0.5 

0.786 

Earnines 

Average  hourly  wage  (%) 

Less  than  $5.00 

8.3 

11.1 

-2.8 

0.230 

$5.00  - $6.99 

20.6 

26.7 

-6.1  * 

0.081 

$7.00  - $8.99 

26.7 

18.6 

8.1  ** 

0.018 

$9.00  or  more 

12.2 

11.2 

1.0 

0.708 

Average  weekly  earnings  ($) 

167 

159 

8 

0.503 

Total  earnings  per  week  (%) 

Less  than  $200 

23.7 

25.8 

-2.1 

0.554 

$201 -$300 

28.6 

29.7 

-1.1 

0.770 

$301-$500 

13.9 

9.7 

4.2 

0.112 

$500  or  more 

1.6 

2.4 

-0.8 

0.483 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

20.0 

16.0 

4.1 

0.194 

Paid  vacation 

29.3 

29.9 

-0.6 

0.875 

Paid  holidays  other  than  Christmas  and  New  Year 

28.9 

26.2 

2.7 

0.451 

Dental  benefits 

14.5 

12.6 

2.0 

0.481 

A retirement  plan 

11.8 

7.8 

4.0  * 

0.095 

A health  plan  or  medical  insurance 

18.5 

15.6 

2.9 

0.334 

(continued) 
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Appendix  Table  B.4  (continued) 


Outcome 


ERA  Control  Difference 
Group  Group  (Impact) P -Value 


Schedule1*  (%) 


Regular 

38.9 

39.7 

-0.8 

0.846 

Split  shift 

1.0 

1.4 

-0.4 

0.626 

Irregular 

3.1 

2.5 

0.6 

0.667 

Evening  shift 

10.5 

6.5 

3.9  * 

0.087 

Night  shift 

2.7 

5.1 

-2.4 

0.140 

Rotating  shift 

9.0 

10.7 

-1.7 

0.493 

Other  schedule 

0.6 

0.8 

-0.2 

0.740 

Odd  job 

2.1 

0.9 

1.2 

0.241 

Job  skills 


Percentage  reporting  that  job  requires  each  at  least  monthly 


Reading  and  writing  skills 

44.5 

43.1 

1.5 

0.725 

Work  with  computers 

16.6 

19.0 

-2.4 

0.448 

Arithmetic  skills 

31.0 

34.0 

-3.1 

0.429 

Customer  contact 

54.9 

60.7 

-5.8 

0.157 

Sample  size  (total  = 598) 

306 

292 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.5 

Impacts  on  Employment  Retention,  Advancement,  and  TANF  Receipt 
in  the  Cumulative  Follow-Up  Period  (Years  1-4),  by  Study  Cohort: 


Chicago 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Earlv  cohort 

Employment  measures 

Ever  employed  (%) 

79.3 

78.3 

1.0 

0.657 

Average  quarterly  employment  (%) 

55.3 

50.9 

4.3  ** 

0.026 

TIad  employment  spell  of  at  least  4 quarters  (%) 

65.3 

60.2 

5 1 ** 

0.049 

t 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

7,055 

6,122 

933  ** 

0.011 

t 

Average  annual  earnings  of  $10,000  or  more  (%) 

32.4 

28.3 

4.1 

0.137 

Quarters  with  earnings  of  $3,500  or  more  (%) 

22.2 

17.8 

44  ** 

0.011 

Public  assistance  and  income  (Years  1-3) 

Average  annual  TANF  received  ($) 

790 

1,007 

S)\~j  *** 

0.001 

Average  annual  food  stamps  received  ($) 

3,951 

3,853 

98 

0.358 

Average  annual  income  ($) 

11,425 

10,772 

653  * 

0.059 

Sample  size  (total  = 866) 

430 

436 

Late  cohort 

Employment  measures 

Ever  employed  (%) 

81.6 

80.4 

1.1 

0.625 

Average  quarterly  employment  (%) 

54.6 

53.8 

0.8 

0.715 

Had  employment  spell  of  at  least  4 quarters  (%) 

64.1 

65.8 

-1.7 

0.553 

t 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

6,872 

6,861 

11 

0.977 

t 

Average  annual  earnings  of  $10,000  or  more  (%) 

30.0 

30.3 

-0.3 

0.920 

Quarters  with  earnings  of  $3,500  or  more  (%) 

21.0 

20.1 

0.9 

0.605 

Public  assistance  and  income  (Years  1-3) 

Average  annual  TANF  received  ($) 

768 

1,012 

-244  * * * 

0.000 

Average  annual  food  stamps  received  ($) 

4,272 

4,065 

207  * 

0.069 

Average  annual  income  ($) 

11,505 

11,468 

37 

0.919 

Sample  size  (total  = 862) 

424 

438 

SOURCE:  MDRC  calculations  from  state  unemployment  insurance  (UI)  wage  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups 
arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
different  subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  ft  = 5 percent;  f 
= 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.6 

Impacts  on  Current  Job  Characteristics  and  Advancement 
at  the  42-Month  Survey  for  the  Early  Cohort: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

91.2 

88.0 

3.3 

0.207 

Currently  employed 

65.8 

63.2 

2.7 

0.518 

No  longer  employed 

25.5 

24.8 

0.6 

0.868 

Hourlv  wase 

Average  hourly  wage  (%) 

Less  than  $5.00 

7.8 

10.3 

-2.5 

0.343 

$5 .00 -$6.99 

10.8 

15.6 

-4.8 

0.123 

$7.00-  $8.99 

21.4 

20.0 

1.5 

0.691 

$9 .00  or  more 

23.1 

17.3 

5.9 

0.110 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

8.3 

14.2 

-5  9 ** 

0.033 

30-34 

11.2 

11.5 

-0.4 

0.892 

35-44 

41.3 

32.4 

9 0** 

0.031 

tt 

45  or  more 

4.9 

4.8 

0.0 

0.984 

Schedule 

Typical  schedule  (%) 

Regular 

34.4 

33.1 

1.3 

0.752 

t 

Evening  shift 

7.2 

7.3 

-0.1 

0.966 

Other  schedule 

23.5 

20.5 

2.9 

0.416 

Workweek  included  at  least  1 weekend  day  (%) 

29.8 

30.6 

-0.8 

0.847 

Benefits 

Employer  offers  (%) 

Sick  days  with  full  pay 

25.3 

18.1 

y 9 ** 

0.042 

Paid  vacation 

35.7 

27.2 

8.5  ** 

0.030 

Paid  holidays  other  than  Christmas  and  New  Year 

29.6 

23.4 

6.2  * 

0.093 

A health  plan  or  medical  insurance 

24.1 

14.4 

90  * * * 

0.004 

ttt 

None  of  the  above 

23.7 

28.9 

-5.2 

0.171 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

47.3 

49.6 

-2.3 

0.594 

Receive  respect  from  coworkers 

53.3 

50.0 

3.3 

0.437 

Receive  proper  equipment  needed  to  do  job 

58.3 

56.2 

2.0 

0.635 

t 

Are  allowed  to  contribute  ideas 

51.9 

47.6 

4.3 

0.318 

tt 

Can  count  on  keeping  job 

25.9 

25.2 

0.7 

0.852 

Think  job  requires  a lot  of  responsibility 

59.7 

57.2 

2.5 

0.558 

Think  job  is  physically  demanding 

28.1 

27.0 

1.1 

0.785 

Risk  health  or  safety 

21.7 

21.7 

-0.1 

0.985 

(continued) 
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Appendix  Table  B.6  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Are  members  of  labor  union  (%) 

19.4 

13.3 

6.1  * 

0.053 

Have  possibility  of  a promotion  (%) 

36.7 

32.3 

4.3 

0.293 

Duties/reauirements.  at  least  once  Der  month  t%  renortinsl 

Reading  and  writing  skills 

48.9 

39.2 

9.6  ** 

0.026 

tt 

Work  with  computers 

21.3 

19.7 

1.6 

0.632 

Arithmetic  skills 

27.6 

25.7 

1.9 

0.613 

Customer  contact 

58.7 

57.1 

1.6 

0.702 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

25.5 

24.8 

0.6 

0.868 

0-15 

15.4 

19.6 

-4.2 

0.198 

16-30 

20.1 

15.0 

5.1 

0.116 

31-45 

12.6 

11.8 

0.8 

0.777 

46  or  more 

17.6 

16.9 

0.7 

0.831 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

29.3 

23.7 

5.7 

0.133 

Ever  received  a promotion  (%) 

14.0 

11.3 

2.8 

0.328 

Found  a different  job  while  working  (%) 

19.2 

18.6 

0.5 

0.875 

Left  a job  to  go  into  a higher-paying  one  (%) 

17.7 

14.9 

2.9 

0.367 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

17.8 

22.8 

-5.0 

0.151 

t 

Earnings 

20.9 

20.0 

0.9 

0.795 

Benefits 

13.1 

10.5 

2.6 

0.344 

Number  of  hours 

20.8 

19.5 

1.3 

0.717 

Start  and  end  of  workday 

20.4 

21.1 

-0.7 

0.842 

Commuting  time 

14.4 

22.0 

-7.6  ** 

0.022 

tt 

Job  security 

17.2 

19.1 

-1.9 

0.569 

Opportunity  to  advance 

15.8 

14.4 

1.4 

0.650 

Sample  size  (total  = 545) 

280 

265 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
different  subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  ft  = 5 percent; 
t = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.7 

Impacts  on  Current  Job  Characteristics  and  Advancement 
at  the  42-Month  Survey  for  the  Late  Cohort: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

94.5 

91.7 

2.8 

0.240 

Currently  employed 

67.2 

73.0 

-5.8 

0.159 

No  longer  employed 

27.3 

18.7 

8.6  ** 

0.025 

Hourlv  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

7.9 

7.9 

0.0 

0.997 

$5.00-  $6.99 

13.7 

16.7 

-3.0 

0.396 

$7.00-  $8.99 

21.0 

20.5 

0.4 

0.915 

$9 .00  or  more 

23.5 

26.0 

-2.5 

0.551 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

13.5 

13.9 

-0.4 

0.891 

30-34 

13.0 

16.1 

-3.1 

0.343 

35-44 

34.1 

39.4 

-5.4 

0.230 

tt 

45  or  more 

6.5 

4.0 

2.5 

0.222 

Schedule 

Typical  schedule  (%) 

Regular 

33.1 

42.3 

_9  2 ** 

0.039 

t 

Evening  shift 

7.0 

7.3 

-0.3 

0.898 

Other  schedule 

24.2 

21.4 

2.8 

0.476 

Workweek  included  at  least  1 weekend  day  (%) 

32.3 

34.2 

-1.9 

0.667 

Benefits 

Employer  offers  (%) 

Sick  days  with  fall  pay 

26.0 

24.3 

1.8 

0.661 

Paid  vacation 

35.2 

35.3 

-0.1 

0.987 

Paid  holidays  other  than  Christmas  and  New  Year 

34.0 

35.1 

-1.1 

0.794 

A health  plan  or  medical  insurance 

21.0 

25.1 

-4.1 

0.288 

ttt 

None  of  the  above 

24.3 

27.9 

-3.6 

0.362 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

54.5 

61.2 

-6.7 

0.134 

Receive  respect  from  coworkers 

57.8 

62.3 

-4.6 

0.303 

Receive  proper  equipment  needed  to  do  job 

59.0 

67.8 

-8.8  ** 

0.046 

t 

Are  allowed  to  contribute  ideas 

53.9 

63.6 

_g  ~j  ** 

0.030 

tt 

Can  count  on  keeping  job 

28.1 

33.3 

-5.1 

0.228 

Think  job  requires  a lot  of  responsibility 

60.8 

67.7 

-6.9 

0.113 

Think  job  is  physically  demanding 

30.1 

36.9 

-6.8 

0.119 

Risk  health  or  safety 

26.9 

28.1 

-1.2 

0.772 

(continued) 
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Appendix  Table  B.7  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Are  members  of  labor  union  (%) 

17.2 

11.7 

5.4  * 

0.094 

Elave  possibility  of  a promotion  (%) 

37.7 

42.1 

-4.4 

0.328 

Duties/reauirements.  at  least  once  Der  month  t%  renortinsl 

Reading  and  writing  skills 

47.5 

50.9 

-3.5 

0.452 

tt 

Work  with  computers 

24.7 

23.8 

1.0 

0.804 

Arithmetic  skills 

35.9 

36.6 

-0.7 

0.872 

Customer  contact 

59.5 

63.4 

-3.9 

0.386 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

27.3 

18.7 

8.6  ** 

0.025 

0-15 

14.1 

19.1 

-5.0 

0.137 

16-30 

19.1 

21.7 

-2.7 

0.472 

31-45 

14.1 

11.5 

2.7 

0.382 

46  or  more 

19.9 

20.5 

-0.7 

0.859 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

36.1 

28.5 

7.6  * 

0.080 

Ever  received  a promotion  (%) 

17.1 

11.4 

5.7  * 

0.076 

Found  a different  job  while  working  (%) 

23.8 

22.2 

1.5 

0.692 

Left  a job  to  go  into  a higher-paying  one  (%) 

23.9 

23.3 

0.7 

0.868 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

26.9 

22.0 

4.8 

0.228 

t 

Earnings 

28.5 

21.7 

6.7  * 

0.097 

Benefits 

19.1 

16.9 

2.1 

0.553 

Number  of  hours 

23.7 

15.9 

y g ** 

0.034 

Start  and  end  of  workday 

24.3 

20.0 

4.3 

0.267 

Commuting  time 

19.8 

17.7 

2.2 

0.556 

tt 

Job  security 

24.2 

18.0 

6.2 

0.103 

Opportunity  to  advance 

19.5 

21.1 

-1.5 

0.681 

Sample  size  (total  = 478) 

241 

237 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
different  subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  ft  = 5 percent;  f = 
10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.8 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Los  Angeles  RFS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P -Value 

Contacts 

Ever  had  contact  with  staff/ employment  program  (%) 

54.0 

50.5 

3.5 

0.304 

Average  number  of  contacts  with  staff 

7.3 

6.2 

1.1 

0.252 

Talked  with  staff  in  past  4 weeks  (%) 

20.3 

18.9 

1.4 

0.605 

Staff  ever  talked  with  respondent's  employer  (%) 

9.7 

10.8 

-1.1 

0.602 

Services 

Received  help  with  support  services  (%) 

59.1 

52.6 

6.5  * 

0.055 

Received  help  with  basic  needs  (%) 

44.2 

38.7 

5.5 

0.110 

Received  help  with  public  benefits  (%) 

62.1 

55.1 

* 

* 

o 

0.039 

Received  help  with  job  preparation  (%) 

35.4 

29.4 

6.1  * 

0.059 

Received  any  help  with  retention/advancement  (%) 

21.6 

17.5 

4.1 

0.134 

Finding  a better  job  while  working 

7.9 

5.8 

2.2 

0.221 

Other  activities  while  working 

7.9 

5.5 

2.4 

0.159 

Career  assessment 

14.4 

9.9 

4.5  ** 

0.045 

Dealing  with  problems  on  the  job 

5.5 

3.7 

1.8 

0.221 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

4.0 

3.8 

0.2 

0.867 

ParticiDation 

Participated  in  any  activity  (%) 

72.0 

68.1 

3.9 

0.226 

Participated  in  any  employment-related  activity3  (%) 

62.8 

56.8 

6.0  * 

0.079 

Participated  in  a job  search  activity  (%) 

60.4 

55.2 

5.2 

0.130 

Group  job  search/job  club 

41.3 

41.7 

-0.4 

0.915 

Individual  job  search 

45.2 

41.9 

3.3 

0.341 

Participated  in  an  education/training  activity  (%) 

35.9 

34.2 

1.7 

0.602 

ABE/GED  and  ESL 

13.7 

12.0 

1.8 

0.433 

College  courses 

16.2 

19.4 

-3.2 

0.226 

Vocational  training 

12.5 

9.7 

2.8 

0.209 

Participated  in  unpaid  work/subsidized  employment  (%) 

7.1 

6.8 

0.3 

0.849 

Participated  in  an  education  activity  while  working  (%) 

19.3 

18.2 

1.2 

0.667 

Participated  in  an  employment  activity  while  working  (%) 

12.5 

9.2 

3.3 

0.123 

Average  number  of  weeks  participating  in 

Job  search  activities 

5.4 

4.6 

0.8 

0.219 

Education/ training  activities 

7.0 

7.1 

-0.1 

0.906 

Unpaid  work/subsidized  employment 

0.9 

1.4 

-0.5 

0.285 

Sample  size  (total  = 848) 

428 

420 

(continued) 
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Appendix  Table  B.8  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.9 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Standard 

Error 

Fo11ow-Ud  Year  1 

Ever  employed  (%) 

84.7 

84.3 

0.4 

0.664 

0.9 

Average  quarterly  employment  (%) 

68.0 

67.5 

0.5 

0.627 

0.9 

Employed  every  quarter  (%) 

50.5 

48.5 

2.0 

0.106 

1.3 

Employed  with  annual  earnings  over  $10,000  (%) 

42.0 

41.0 

1.0 

0.422 

1.2 

Annual  earnings  ($) 

9,139 

9,037 

103 

0.611 

202 

Ever  received  TANF  (%) 

93.9 

93.0 

0.9 

0.154 

0.7 

Ever  received  food  stamps  (%) 

91.3 

90.8 

0.5 

0.494 

0.7 

Total  income  ($) 

15,590 

15,358 

232 

0.188 

176 

Fo11ow-Ud  Year  2 

Ever  employed  (%) 

76.1 

77.1 

-1.0 

0.343 

1.1 

Average  quarterly  employment  (%) 

61.8 

62.9 

-1.1 

0.275 

1.0 

Employed  every  quarter  (%) 

46.5 

47.3 

-0.7 

0.560 

1.3 

Employed  with  annual  earnings  over  $10,000  (%) 

41.3 

43.0 

-1.7 

0.167 

1.3 

Annual  earnings  ($) 

9,599 

9,799 

-200 

0.417 

246 

Ever  received  TANF  (%) 

70.5 

66.5 

4 o *** 

0.001 

1.2 

Ever  received  food  stamps  (%) 

70.4 

67.3 

3 o *** 

0.010 

1.2 

Total  income  ($) 

14,945 

14,917 

28 

0.900 

225 

FoIIow-Ud  Year  3 

Ever  employed  (%) 

74.2 

75.4 

-1.3 

0.254 

1.1 

Average  quarterly  employment  (%) 

61.0 

62.2 

-1.2 

0.250 

1.1 

Employed  every  quarter  (%) 

46.6 

47.1 

-0.5 

0.727 

1.3 

Employed  with  annual  earnings  over  $10,000  (%) 

44.0 

44.1 

-0.1 

0.920 

1.3 

Annual  earnings  ($) 

10,762 

10,677 

85 

0.763 

282 

Ever  received  TANF  (%) 

55.9 

53.8 

2.1  * 

0.099 

1.3 

Ever  received  food  stamps  (%) 

57.1 

54.7 

2.5  * 

0.052 

1.3 

Total  income  ($) 

15,070 

14,920 

150 

0.570 

264 

Sample  size  (total  = 5,700) 

2,857 

2,843 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.10 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-3): 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Enmlovment  measures 

Number  of  quarters  employed  at  jobs  held  in  the 

quarter  after  random  assignment3 

3.6 

3.5 

0.0 

0.825 

Average  quarterly  employment  (%) 

63.6 

64.2 

-0.6 

0.453 

Percentage  of  quarters  employed  (%) 

Never  employed 

7.7 

8.5 

-0.8 

0.279 

1-25 

14.7 

13.3 

1.4 

0.120 

26-50 

15.1 

14.0 

1.0 

0.274 

51-75 

18.3 

18.8 

-0.6 

0.590 

76-100 

44.3 

45.4 

-1.1 

0.367 

Employed  entire  follow-up  period  (%) 

29.1 

28.3 

0.8 

0.473 

Had  employment  spell  of  at  least  4 quarters  (%) 

70.5 

70.6 

-0.2 

0.893 

Average  number  of  employers  during  follow-up  period 

3.1 

3.1 

0.0 

0.645 

Number  of  employers  (%) 

Never  employed 

7.7 

8.5 

-0.8 

0.279 

1 to  2 

41.9 

40.6 

1.3 

0.325 

3 to  4 

28.0 

28.1 

-0.1 

0.932 

5 to  8 

19.3 

19.1 

0.2 

0.869 

More  than  8 

3.2 

3.8 

-0.6 

0.214 

Average  number  of  quarters  in  first  employment  spell 

6.1 

5.9 

0.1 

0.256 

Number  of  quarters  until  first  employment  spell 

1.5 

1.6 

-0.1 

0.520 

Average  number  of  employment  spells 

1.4 

1.4 

0.0 

0.414 

Average  length,  longest  employment  spell  (quarters) 

6.7 

6.7 

0.0 

0.931 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

7.7 

8.5 

-0.8 

0.279 

1 to  2 

15.3 

13.2 

2.1  ** 

0.024 

3 to  4 

13.8 

14.2 

-0.4 

0.636 

5 to  8 

23.8 

24.8 

-1.0 

0.393 

9 to  12 

39.4 

39.3 

0.1 

0.952 

(continued) 
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Appendix  Table  B.10  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Average  number  of  unemployment  spells 

1.1 

1.1 

0.0 

0.920 

Length  of  longest  unemployment  spell,  in  quarters 

3.8 

3.7 

0.1 

0.556 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

29.1 

28.3 

0.8 

0.473 

1 to  2 

20.6 

22.7 

-2.0  * 

0.061 

3 to  4 

15.2 

15.3 

-0.1 

0.959 

5 to  8 

18.3 

17.1 

1.1 

0.258 

9 to  12 

16.8 

16.7 

0.1 

0.898 

Earnings  and  advancement  measures 

Average  annual  earnings  (S) 

9,833 

9,837 

-4 

0.985 

Average  annual  earnings  (%) 

$0 

7.7 

8.5 

-0.8 

0.279 

$l-$  1,999 

15.9 

14.5 

1.5 

0.117 

$2,000-$4,999 

14.3 

14.1 

0.3 

0.780 

S5,000-$9,999 

20.0 

20.2 

-0.2 

0.844 

$10,000-514,999 

16.5 

17.2 

-0.7 

0.510 

$15,000-519,999 

12.0 

12.4 

-0.5 

0.598 

$20,000  or  higher 

13.5 

13.1 

0.3 

0.696 

Average  annual  earnings  of  $ 1 0,000  or  more  (%) 

42.0 

42.8 

-0.8 

0.532 

Quarters  with  earnings  of  $3,500  or  more  (%) 

32.9 

33.6 

-0.7 

0.427 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

7.7 

8.5 

-0.8 

0.279 

No  quarters  with  earnings  above  $3,500 

26.0 

24.7 

1.4 

0.218 

1 to  2 

16.7 

16.9 

-0.2 

0.871 

3 to  4 

12.0 

10.8 

1.3 

0.134 

5 to  8 

17.2 

18.5 

-1.3 

0.216 

9 to  12 

20.3 

20.8 

-0.5 

0.627 

Comparison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  3 

Not  employed  in  either  Year  1 or  Year  3 

8.6 

9.4 

-0.8 

0.285 

Earnings  decreased  (%) 

No  longer  employed 

17.2 

15.2 

2.0  ** 

0.036 

Earnings  decreased  by  less  than  $250 

3.1 

3.5 

-0.5 

0.341 

Earnings  decreased  by  $250  or  more 

18.9 

19.3 

-0.4 

0.671 

Earnings  increased  (%) 

Became  employed 

6.8 

6.4 

0.4 

0.560 

Earnings  increased  by  less  than  $250 

3.4 

3.9 

-0.4 

0.388 

Earnings  increased  by  $250  or  more 

41.8 

42.2 

-0.4 

0.752 

Average  earnings  per  quarter  employed  ($) 

3,866 

3,829 

37 

Sample  size  (total  = 5,700) 

2,857 

2,843 

(continued) 
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Appendix  Table  B.10  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES  :The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control 
groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only 
for  sample  members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of 
program  group  and  control  group  members  who  were  employed,  any  differences  in  outcomes  may  not 
necessarily  be  attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took 
place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after 
random  assignment,  even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given 
to  those  who  were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.ll 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

81.1 

78.0 

3.1 

0.263 

Currently  employed 

58.6 

53.1 

5.5 

0.107 

No  longer  employed 

22.5 

24.9 

-2.4 

0.420 

Current  working  status  (%) 

Full  time 

48.0 

43.5 

4.5 

0.185 

Part  time 

10.6 

9.7 

1.0 

0.649 

Currently  employed  at  a "good  job"a(%) 

27.3 

25.3 

2.0 

0.498 

Hours 

Average  hours  per  week 

21.3 

18.9 

2.5  * 

0.063 

Total  hours  per  week  (%) 

Less  than  30 

10.6 

9.7 

1.0 

0.649 

30-34 

6.8 

8.1 

-1.4 

0.460 

3544 

33.7 

29.0 

4.6 

0.143 

45  or  more 

6.8 

4.7 

2.1 

0.191 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

3.0 

1.9 

1.1 

0.317 

$5.00 -$6.99 

7.3 

5.2 

2.2 

0.197 

$7.00  - $8.99 

21.0 

23.3 

-2.3 

0.428 

$9.00  or  more 

27.2 

22.7 

4.5 

0.116 

Average  weekly  earnings  ($) 

199 

181 

19 

0.187 

Total  earnings  per  week  (%) 

Less  than  $200 

9.3 

7.9 

1.4 

0.478 

$201-$300 

17.6 

16.6 

1.0 

0.698 

$301-$500 

24.1 

20.5 

3.6 

0.207 

$500  or  more 

7.6 

8.2 

-0.6 

0.767 

Benefits 

Currendy  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

21.4 

20.1 

1.4 

0.625 

Paid  vacation 

27.4 

27.8 

-0.3 

0.916 

Paid  holidays  other  than  Christmas  and  New  Year 

28.6 

26.9 

1.7 

0.579 

Dental  benefits 

21.1 

19.7 

1.4 

0.618 

A retirement  plan 

19.3 

16.0 

3.3 

0.205 

A health  plan  or  medical  insurance 

25.0 

22.6 

2.5 

0.403 

(continued) 
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Appendix  Table  B.ll  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Schedule"  (%) 

Regular 

36.3 

31.1 

5.2 

0.103 

Split  shift 

1.9 

2.6 

-0.6 

0.543 

Irregular 

2.5 

3.0 

-0.5 

0.659 

Evening  shift 

6.5 

5.1 

1.4 

0.404 

Night  shift 

3.2 

3.9 

-0.8 

0.549 

Rotating  shift 

6.7 

4.9 

1.9 

0.256 

Other  schedule 

0.5 

0.0 

0.4 

0.201 

Odd  job 

Job  skills 

0.9 

2.4 

-1.6  * 

0.082 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

42.4 

42.6 

-0.2 

0.952 

Work  with  computers 

26.8 

24.2 

2.6 

0.378 

Arithmetic  skills 

25.2 

27.0 

-1.9 

0.543 

Customer  contact 

49.6 

44.9 

4.7 

0.173 

Sample  size  (total  = 848) 428 420 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  perecent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.12 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Riverside  Phase  2 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact  P- Value 

Impact  P-Value 

Contacts 


Ever  had  contact  with  staff  employment  program2  (%) 

58.2 

58.6 

50.2 

7.9  * 

0.081 

8.3  * 

0.069 

Average  number  of  contacts  with  staff 

9.8 

8.5 

8.0 

1.8 

0.271 

0.5 

0.762 

Talked  with  staff  in  past  4 weeks  (%) 

23.7 

23.6 

21.8 

1.9 

0.617 

1.8 

0.638 

Staff  ever  talked  with  respondent's  employer  (%) 

7.8 

10.2 

7.0 

0.8 

0.745 

3.2 

0.222 

Services 

Received  help  with  support  services  (%) 

61.2 

64.0 

59.8 

1.4 

0.744 

4.2 

0.336 

Received  help  with  basic  needs  (%) 

35.3 

38.3 

42.3 

-7.0 

0.122 

-4.0 

0.379 

Received  help  with  public  benefits  (%) 

67.4 

62.2 

64.7 

2.7 

0.543 

-2.6 

0.564 

Received  help  with  job  preparation  (%) 

47.8 

45.8 

39.2 

8.6  * 

0.059 

6.7 

0.148 

Received  any  help  with  retention/advancement  (%) 

24.2 

28.9 

21.5 

2.6 

0.513 

7.4  * 

0.066 

Finding  a better  job  while  working 

9.9 

11.4 

9.5 

0.4 

0.891 

2.0 

0.488 

Other  activities  while  working 

4.2 

7.7 

8.7 

-4.4  * 

0.057 

-1.0 

0.663 

Career  assessment 

13.5 

15.3 

10.5 

3.0 

0.330 

4.8 

0.124 

Dealing  with  problems  on  the  job 

4.7 

6.7 

7.1 

-2.4 

0.279 

-0.5 

0.831 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

8.1 

8.5 

9.5 

-1.5 

0.577 

-1.0 

0.701 

Participation 

Participated  in  any  activity  (%) 

75.3 

78.6 

73.3 

1.9 

0.620 

5.2 

0.184 

Participated  in  any  employment-related  activity  (%) 

64.7 

65.7 

61.4 

3.3 

0.446 

4.3 

0.331 

Participated  in  a job  search  activity  (%) 

62.1 

64.4 

60.3 

1.9 

0.675 

4.1 

0.356 

Group  job  search/job  club 

38.6 

44.9 

44.6 

-6.0 

0.189 

0.3 

0.950 

Individual  job  search 

50.7 

51.2 

48.6 

2.1 

0.649 

2.7 

0.564 

(continued) 
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Appendix  Table  B.12  (continued) 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Training  Focused 
Group 

Impact  P-Value 

Work  Plus 
Group 

Impact  P -Value 

Participated  in  an  education/training  activity  (%) 

41.4 

37.3 

31.9 

9.5  ** 

0.033 

5.4 

0.225 

ABE/GED  and  ESL 

16.1 

17.6 

9.0 

7 2** 

0.016 

g y *** 

0.004 

College  courses 

19.9 

18.1 

17.3 

2.6 

0.455 

0.8 

0.819 

Vocational  training 

12.2 

8.2 

8.3 

3.8 

0.158 

-0.1 

0.974 

Participated  in  unpaid  work/subsidized  employment  (%) 

9.6 

3.6 

6.2 

3.4 

0.137 

-2.6 

0.244 

Participated  in  an  education  activity  while  working  (%) 

26.0 

29.7 

22.4 

3.6 

0.376 

7.3  * 

0.074 

Participated  in  an  employment  activity  while  working  (%) 

13.4 

15.3 

15.2 

-1.8 

0.582 

0.1 

0.981 

Average  number  of  weeks  participating  in 

Job  search  activities 

5.4 

4.0 

4.9 

0.5 

0.560 

-0.9 

0.289 

Education/training  activities 

6.9 

8.7 

5.1 

1.8 

0.148 

3.6  *** 

0.005 

Unpaid  work/subsidized  employment 

1.2 

0.4 

1.5 

-0.3 

0.563 

-1  \ ** 

0.038 

Sample  size  (total  =712) 

234 

237 

241 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discuseed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the-job  training,  or  who  had  an  unpaid  or 
subsidized  job. 


The  Employment  Retention  and  Advancement  Project 

Appendix  Table  B.13 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Riverside  Phase  2 


Training  Focused  Work  Plus 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Impact 

Group 

P-Value 

Standard 

Error 

Impact 

Group 
P -Value 

Standard 

Error 

Follow-Up  Year  1 

Ever  employed  (%) 

87.3 

88.7 

89.5 

-2.2 

0.165 

1.6 

-0.8 

0.570 

1.4 

Average  quarterly  employment  (%) 

68.0 

70.1 

72.5 

.4  5 ** 

0.011 

1.8 

-2.4 

0.119 

1.5 

Employed  every  quarter  (%) 

47.3 

48.9 

51.9 

-4.6  * 

0.069 

2.5 

-3.0 

0.172 

2.2 

Employed  with  annual  earnings  over  $10,000  (%) 

35.3 

35.9 

37.1 

-1.9 

0.435 

2.4 

-1.3 

0.536 

2.1 

Annual  earnings  ($) 

8,080 

8,079 

8,344 

-263 

0.458 

355 

-265 

0.387 

306 

Ever  received  TANF  (%) 

86.6 

86.6 

84.1 

2.6 

0.148 

1.8 

2.6  * 

0.094 

1.5 

Ever  received  food  stamps  (%) 

81.7 

83.5 

80.3 

1.4 

0.437 

1.8 

3 2** 

0.043 

1.6 

Total  income  ($) 

12,787 

12,665 

12,764 

23 

0.943 

321 

-99 

0.719 

277 

Follow-Up  Year  2 

Ever  employed  (%) 

76.5 

75.5 

77.2 

-0.7 

0.737 

2.2 

-1.6 

0.378 

1.9 

Average  quarterly  employment  (%) 

59.5 

60.0 

61.5 

-2.0 

0.345 

2.1 

-1.5 

0.407 

1.8 

Employed  every  quarter  (%) 

41.6 

42.8 

44.8 

-3.1 

0.212 

2.5 

-1.9 

0.371 

2.2 

Employed  with  annual  earnings  over  $10,000  (%) 

38.4 

35.9 

38.4 

0.1 

0.980 

2.4 

-2.5 

0.236 

2.1 

Annual  earnings  ($) 

8,760 

8,138 

8,308 

452 

0.310 

445 

-170 

0.657 

383 

Ever  received  TANF  (%) 

53.2 

52.8 

51.4 

1.8 

0.477 

2.5 

1.4 

0.521 

2.2 

Ever  received  food  stamps  (%) 

55.5 

57.9 

56.4 

-1.0 

0.694 

2.5 

1.5 

0.494 

2.1 

Total  income  ($) 

12,301 

11,515 

11,606 

695 

0.102 

425 

-92 

0.803 

366 

(continued) 
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Appendix  Table  B.13  (continued) 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Training  Focused 
Group 

Standard 

Impact  P-Value  Error 

Impact 

Work  Plus 
Group 

P-Value 

Standard 

Error 

Fo11ow-Ud  Year  3 

Ever  employed  (%) 

73.7 

71.2 

72.2 

1.5 

0.496 

2.3 

-1.0 

0.603 

2.0 

Average  quarterly  employment  (%) 

57.7 

56.4 

58.2 

-0.4 

0.844 

2.1 

-1.8 

0.329 

1.8 

Employed  every  quarter  (%) 

41.2 

40.7 

43.0 

-1.8 

0.479 

2.5 

-2.2 

0.300 

2.2 

Employed  with  annual  earnings  over  $10,000  (%) 

38.8 

36.6 

38.7 

0.1 

0.955 

2.4 

-2.1 

0.322 

2.1 

Annual  earnings  ($) 

9,355 

8,524 

9,103 

252 

0.617 

505 

-579 

0.184 

436 

Ever  received  TANF  (%) 

39.5 

39.5 

36.9 

2.6 

0.296 

2.4 

2.5 

0.230 

2.1 

Ever  received  food  stamps  (%) 

44.4 

47.1 

44.7 

-0.4 

0.885 

2.5 

2.4 

0.259 

2.1 

Total  income  ($) 

12,363 

11,311 

11,867 

497 

0.315 

494 

-556 

0.192 

426 

Fo11ow-Ud  Year  4 

Ever  employed  (%) 

70.5 

66.6 

69.3 

1.3 

0.593 

2.4 

-2.7 

0.190 

2.0 

Average  quarterly  employment  (%) 

57.2 

53.7 

56.0 

1.1 

0.607 

2.2 

-2.3 

0.218 

1.9 

Employed  every  quarter  (%) 

43.3 

40.3 

42.8 

0.5 

0.843 

2.5 

-2.5 

0.244 

2.2 

Employed  with  annual  earnings  over  $10,000  (%) 

40.8 

38.4 

38.1 

2.6 

0.283 

2.5 

0.3 

0.904 

2.1 

Annual  earnings  ($) 

9,958 

9,235 

9,736 

222 

0.695 

565 

-502 

0.303 

487 

Ever  received  TANF  (%) 

29.6 

30.5 

29.0 

0.6 

0.782 

2.3 

1.5 

0.453 

2.0 

Ever  received  food  stamps  (%) 

37.7 

38.4 

37.3 

0.5 

0.853 

2.4 

1.1 

0.582 

2.1 

Total  income  ($) 

12,422 

11,660 

12,153 

269 

0.625 

551 

-494 

0.299 

475 

Sample  size  (total  = 3,029) 

744 

1,532 

753 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 
Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1"  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.14 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

Riverside  Phase  2 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact  P-Value 

Impact  P-Value 

Employment  measures 


Number  of  quarters  employed  at  jobs  held 


in  the  quarter  after  random  assignment3 

3.6 

3.5 

3.9 

-0.3 

0.263 

-0.4  * 

0.074 

Average  quarterly  employment  (%) 

60.6 

60.1 

62.1 

-1.4 

0.378 

-2.0 

0.157 

Percentage  of  quarters  employed  (%) 
Never  employed 

6.1 

5.7 

5.0 

1.0 

0.372 

0.6 

0.526 

1-25 

15.0 

18.2 

16.1 

-1.1 

0.552 

2.1 

0.195 

26-50 

18.3 

16.3 

17.6 

0.7 

0.726 

-1.3 

0.445 

51-75 

21.5 

20.3 

21.1 

0.5 

0.820 

-0.8 

0.677 

76-100 

39.2 

39.5 

40.2 

-1.1 

0.669 

-0.7 

0.726 

Employed  entire  follow-up  period  (%) 

19.3 

18.8 

21.7 

-2.3 

0.253 

-2.8 

0.104 

Had  employment  spell  of  at  least  4 quarters  (%) 

73.9 

72.7 

75.3 

-1.5 

0.507 

-2.6 

0.174 

Average  number  of  employers  during  follow-up  period 

3.6 

3.7 

3.9 

-0.3  * 

0.072 

-0.1 

0.356 

Number  of  employers  (%) 
Never  employed 

9.0 

9.0 

7.9 

1.1 

0.441 

1.1 

0.357 

1 to  2 

32.8 

31.8 

32.3 

0.5 

0.850 

-0.5 

0.793 

3 to  4 

28.9 

27.9 

26.8 

2.1 

0.367 

1.2 

0.564 

5 to  8 

23.7 

24.7 

24.0 

-0.2 

0.917 

0.7 

0.699 

More  than  8 

5.6 

6.6 

9.1 

-3  4 *** 

0.009 

-2.5  ** 

0.028 

Average  number  of  quarters  in  first  employment  spell 

6.5 

6.6 

6.9 

-0.4 

0.169 

-0.3 

0.173 

Number  of  quarters  until  first  employment  spell 

1.6 

1.5 

1.4 

0.2 

0.207 

0.1 

0.510 

(continued) 
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Appendix  Table  B.14  (continued) 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact 

P-Value 

Impact 

P-Value 

Average  annual  earnings  of  $10,000  or  more  (%) 

37.5 

37.0 

35.7 

1.9 

0.440 

1.3 

0.530 

Quarters  with  earnings  of  $3,500  or  more  (%) 
Number  of  quarters  earning  above  $3,500  (%) 

30.2 

27.4 

28.4 

1.8 

0.240 

-1.0 

0.468 

Never  employed 

6.1 

5.7 

5.0 

1.0 

0.372 

0.6 

0.526 

No  quarters  with  earnings  above  $3,500 

25.1 

30.3 

27.5 

-2.5 

0.279 

2.8 

0.159 

1 to  2 

16.2 

14.3 

18.3 

-2.1 

0.270 

_4  0 ** 

0.013 

3 to  4 

10.4 

10.5 

10.3 

0.0 

0.991 

0.1 

0.940 

5 to  8 

16.9 

17.1 

15.1 

1.8 

0.352 

2.0 

0.225 

9 to  12 

12.3 

12.8 

12.4 

-0.1 

0.937 

0.4 

0.785 

13  to  16 

13.1 

9.4 

11.3 

1.8 

0.245 

-1.9 

0.169 

Comparison  of  auarter  with  highest  earnings  in  Year  1 
and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 
Earnings  decreased  (%) 

7.4 

7.6 

5.7 

1.7 

0.179 

1.9  * 

0.086 

No  longer  employed 

22.0 

25.8 

25.0 

-3.0 

0.176 

0.8 

0.693 

Earnings  decreased  by  less  than  $250 

3.2 

1.7 

2.1 

1.1 

0.138 

-0.4 

0.549 

Earnings  decreased  by  $250  or  more 

16.3 

18.0 

18.4 

-2.1 

0.286 

-0.4 

0.814 

Earnings  increased  (%) 

Became  employed 

5.3 

3.7 

4.8 

0.5 

0.647 

-1.1 

0.211 

Earnings  increased  by  less  than  $250 

3.4 

2.4 

3.3 

0.1 

0.949 

-1.0 

0.196 

Earnings  increased  by  $250  or  more 

42.2 

40.8 

40.4 

1.8 

0.464 

0.4 

0.844 

Average  earnings  per  quarter  employed  ($) 

3,729 

3,536 

3,576 

1 53 

-40 

Sample  size  (total  = 3,029) 

744 

1,532 

753 

(continued) 
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Appendix  Table  B.14  (continued) 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact 

P-Value 

Impact 

P-Value 

Average  annual  earnings  of  $10,000  or  more  (%) 

37.5 

37.0 

35.7 

1.9 

0.440 

1.3 

0.530 

Quarters  with  earnings  of  $3,500  or  more  (%) 
Number  of  quarters  earning  above  $3,500  (%) 

30.2 

27.4 

28.4 

1.8 

0.240 

-1.0 

0.468 

Never  employed 

6.1 

5.7 

5.0 

1.0 

0.372 

0.6 

0.526 

No  quarters  with  earnings  above  $3,500 

25.1 

30.3 

27.5 

-2.5 

0.279 

2.8 

0.159 

1 to  2 

16.2 

14.3 

18.3 

-2.1 

0.270 

* 

* 

o 

'xf- 

1 

0.013 

3 to  4 

10.4 

10.5 

10.3 

0.0 

0.991 

0.1 

0.940 

5 to  8 

16.9 

17.1 

15.1 

1.8 

0.352 

2.0 

0.225 

9 to  12 

12.3 

12.8 

12.4 

-0.1 

0.937 

0.4 

0.785 

13  to  16 

13.1 

9.4 

11.3 

1.8 

0.245 

-1.9 

0.169 

Conroarison  of  auarter  with  highest  earnings  in  Year  1 
and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

7.4 

7.6 

5.7 

1.7 

0.179 

1.9  * 

0.086 

Earnings  decreased  (%) 

No  longer  employed 

22.0 

25.8 

25.0 

-3.0 

0.176 

0.8 

0.693 

Earnings  decreased  by  less  than  $250 

3.2 

1.7 

2.1 

1.1 

0.138 

-0.4 

0.549 

Earnings  decreased  by  $250  or  more 

16.3 

18.0 

18.4 

-2.1 

0.286 

-0.4 

0.814 

Earnings  increased  (%) 

Became  employed 

5.3 

3.7 

4.8 

0.5 

0.647 

-1.1 

0.211 

Earnings  increased  by  less  than  $250 

3.4 

2.4 

3.3 

0.1 

0.949 

-1.0 

0.196 

Earnings  increased  by  $250  or  more 

42.2 

40.8 

40.4 

1.8 

0.464 

0.4 

0.844 

Average  earnings  per  quarter  employed  ($) 

3,729 

3,536 

3,576 

153 

-40 

Sample  size  (total  = 3,029) 

744 

1,532 

753 

(continued) 


336 


Appendix  Table  B.14  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample  members  who  were  employed. 
Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control  group  members  who  were  employed,  any  differences  in 
outcomes  may  not  necessarily  be  attributable  to  the  ERA  program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discuseed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random  assignment,  even  among  those  who  were 
working  at  these  jobs  previously,  eroes  were  given  to  those  who  were  not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.15 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Riverside  Phase  2 

Training  Work  Training  Focused  Work  Plus 

Focused  Plus  Control  Group Group 

Outcome Group Group  Group Impact P-Value Impact P-Value 


Employment  status 


Ever  employed  since  random  assignment  (%)  90.4 

Currently  employed  58.8 

No  longer  employed  31.6 

Current  working  status  (%) 

Full  time  45.5 

Part  time  13.3 

Currently  employed  at  a "good  job"a  (%)  26.2 

Hours 

Average  hours  per  week  20.5 

Total  hours  per  week  (%) 

Less  than  30  13.3 

30-34  6.7 

35-44  34.6 

45  or  more  4.2 

Earnings 

Average  hourly  wage  (%) 

Less  than  $5.00  2.8 

$5.00  - $6.99  7.8 

$7.00 -$8.99  23.1 

$9.00  or  more  25.1 


93.6 

89.0 

1.4 

0.597 

4.6  * 

0.085 

70.2 

63.0 

-4.2 

0.342 

7.2 

0.102 

23.4 

25.6 

6.0 

0.139 

-2.2 

0.592 

54.0 

50.5 

-5.0 

0.279 

3.5 

0.444 

16.3 

12.6 

0.8 

0.806 

3.7 

0.253 

29.5 

25.2 

1.0 

0.802 

4.3 

0.296 

25.1 

22.6 

-2.1 

0.238 

2.5 

0.151 

16.3 

12.6 

0.8 

0.806 

3.7 

0.253 

12.0 

10.0 

-3.3 

0.227 

2.0 

0.455 

33.6 

33.0 

1.6 

0.715 

0.6 

0.892 

8.4 

7.5 

-3.3 

0.154 

0.9 

0.698 

2.3 

2.1 

0.6 

0.661 

0.2 

0.909 

10.8 

12.9 

-5.1  * 

0.074 

-2.1 

0.456 

36.6 

26.6 

-3.5 

0.402 

10.0  ** 

0.018 

20.6 

21.3 

3.8 

0.315 

-0.7 

0.845 

(continued) 
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Appendix  Table  B.15  (continued) 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Training  Focused 
Group 

Impact  P- Value 

Work  Plus 
Group 

Impact  P- Value 

Average  weekly  earnings  (S) 

190 

215 

200 

-11 

0.548 

15 

0.407 

Total  earnings  per  week  (%) 

Less  than  $200 

12.4 

14.9 

13.2 

-0.9 

0.784 

1.7 

0.602 

S201-S300 

17.9 

21.5 

22.9 

-5.0 

0.177 

-1.5 

0.697 

$301-$500 

22.3 

31.2 

20.3 

2.1 

0.602 

11  o *** 

0.006 

$500  or  more 

6.3 

2.7 

6.6 

-0.3 

0.866 

-4.0  * 

0.054 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

23.1 

26.7 

20.2 

2.9 

0.450 

6.5  * 

0.098 

Paid  vacation 

27.1 

34.3 

30.3 

-3.2 

0.450 

4.0 

0.345 

Paid  holidays  other  than  Christmas  and  New  Year 

23.3 

29.9 

29.3 

-6.0 

0.147 

0.7 

0.875 

Dental  benefits 

20.6 

24.1 

20.2 

0.3 

0.930 

3.9 

0.311 

A retirement  plan 

17.6 

23.4 

20.0 

-2.4 

0.516 

3.4 

0.351 

A health  plan  or  medical  insurance 

24.3 

30.4 

27.5 

-3.1 

0.442 

2.9 

0.480 

Schedule11  (%> 

Regular 

36.4 

42.4 

33.3 

3.1 

0.479 

9.1  ** 

0.040 

Split  shift 

0.5 

1.6 

0.5 

0.0 

0.991 

1.2 

0.171 

Irregular 

5.8 

4.6 

5.3 

0.5 

0.812 

-0.7 

0.730 

Evening  shift 

2.9 

12.0 

7.8 

_4  9 ** 

0.043 

4.3  * 

0.079 

Night  shift 

5.5 

4.4 

2.0 

3.6  ** 

0.044 

2.4 

0.175 

Rotating  shift 

5.5 

3.1 

11.1 

-5.6  ** 

0.015 

-8  0 *** 

0.001 

Other  schedule 

0.9 

1.2 

2.5 

-1.5 

0.176 

-1.2 

0.281 

Odd job 

1.4 

0.9 

0.3 

1.1 

0.213 

0.6 

0.499 

(continued) 
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Appendix  Table  B.15  (continued) 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Training  Focused 
Group 

Impact  P- Value 

Work  Plus 
Group 

Impact  P- Value 

Job  skills 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

44.4 

52.9 

49.7 

-5.3 

0.245 

3.1 

0.496 

Work  with  computers 

27.0 

36.1 

28.4 

-1.5 

0.724 

7.6  * 

0.071 

Arithmetic  skills 

32.8 

43.6 

36.5 

-3.6 

0.409 

7.1 

0.107 

Customer  contact 

50.3 

60.8 

55.2 

-4.9 

0.280 

5.6 

0.224 

Sample  size  (total  =712) 

234 

237 

241 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 perecent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discused  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which  a respondent  works  35  or  more  hours  per 
week  and  either  (1)  pays  S7.00  or  more  per  hour  and  offers  health  insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined  as  one  that  changes  from  day  to  day.  A rotating 
shift  is  one  that  changes  regularly  from  days  to  evenings  to  nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.16 

Impacts  on  Services,  Participation,  and  Receipt  of  Education  Credentials 

at  the  12-Month  Survey: 

Riverside  Phase  2 


Outcome 

Training 

Focused 

Group 

Work 

Plus 

Group 

Control 

Group 

Training  Focused 
Group 

Impact  P-Value 

Work  Plus 
Group 

Impact  P-Value 

No  hiah  school  diploma  or  GED  certificate 

Received  any  help  with  retention/advancement  (%) 

28.1 

33.8 

14.2 

14.0  ** 

0.030 

^g  7 *** 

0.002 

Participated  in  an  education/training  activity  (%) 

48.6 

40.7 

30.8 

g ** 

0.016 

9.9 

0.173 

ABE/GED  and  ESL 

37.9 

36.8 

19.7 

|g  ^ *** 

0.010 

Y]  ^ ** 

0.014 

College  courses 

9.8 

5.0 

7.7 

2.1 

0.595 

-2.7 

0.494 

Vocational  training 

7.7 

8.5 

6.3 

1.4 

0.736 

2.2 

0.586 

Participated  in  an  education  activity  while  working  (%) 

30.7 

26.9 

16.5 

14.2  ** 

0.026 

10.4 

0.101 

Received  an  education  credential  after  random  assignment  (%) 

High  school  diploma 

1.3 

0.8 

0.1 

1.2 

0.335 

0.7 

0.559 

GED  certificate 

1.5 

4.2 

0.7 

0.8 

0.697 

3.5  * 

0.098 

Trade  or  occupational  license  or  certificate 

7.8 

3.8 

4.6 

3.3 

0.341 

-0.7 

0.825 

Sample  size  (total  = 281) 

90 

95 

96 

Hiah  school  diploma  or  GED  certificate 

Received  any  help  with  retention/advancement  (%) 

21.5 

26.6 

25.8 

-4.4 

0.407 

0.8 

0.885 

Participated  in  an  education/training  activity  (%) 

38.1 

32.6 

32.8 

5.3 

0.356 

-0.2 

0.976 

ABE/GED  and  ESL 

2.5 

4.5 

1.5 

1.0 

0.611 

2.9 

0.145 

College  courses 

27.7 

25.0 

24.0 

3.7 

0.482 

1.0 

0.849 

Vocational  training 

14.9 

7.4 

9.3 

5.6 

0.127 

-1.9 

0.610 

Participated  in  an  education  activity  while  working  (%) 

23.9 

29.8 

26.6 

-2.7 

0.611 

3.1 

0.566 

Received  an  education  credential  after  random  assignment  (%) 

High  school  diploma 

0.7 

0.0 

-0.1 

0.8 

0.182 

0.1 

0.898 

GED  certificate 

0.0 

0.0 

0.0 

0.0 

0.000 

0.0 

0.000 

Trade  or  occupational  license  or  certificate 

15.7 

10.0 

15.1 

0.6 

0.880 

-5.1 

0.223 

Sample  size  (total  = 426) 

141 

141 

144 

(continued) 


Appendix  Table  B.16  (continued) 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across  different  subgroups.  Statistical  significance  levels 
are  indicated  as  follows:  fff  = 1 percent;  ff  = 5 percent;  f = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  B.17 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4),  by  Educational  Attainment: 

Riverside  Phase  2 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact 

P- Value 

Impact 

P -Value 

No  hiah  school  diolonia  or  GED  certificate 

Employment  measures 

Ever  employed  (%) 

93.0 

94.2 

95.9 

-2.8 

0.120 

-1.7 

0.284 

Average  quarterly  employment  (%) 

57.6 

57.2 

60.0 

-2.4 

0.331 

-2.8 

0.194 

Had  employment  spell  of  at  least  4 quarters  (%) 

67.9 

69.7 

74.0 

-6.1 

* 0.082 

-4.3 

0.153 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

7,624 

7,434 

7,479 

145 

0.782 

-45 

0.921 

Average  annual  earnings  of  $10,000  or  more  (%) 

31.9 

31.6 

28.5 

3.4 

0.332 

3.2 

0.295 

Quarters  with  earnings  of  $3,500  or  more  (%) 

24.4 

23.2 

22.5 

1.8 

0.391 

0.6 

0.723 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

2,640 

2,233 

2,258 

382 

**  0.022 

-25 

0.859 

Average  annual  food  stamps  received  ($) 

1,520 

1,381 

1,419 

102 

0.309 

-37 

0.666 

Average  annual  income  ($) 

11,785 

11,049 

11,156 

629 

0.215 

-107 

0.806 

Sample  size  (total  = 1,255) 

620 

305 

330 

Hiah  school  dmloma  or  GED  certificate 

Employment  retention  measures 

Ever  employed  (%) 

94.5 

94.7 

94.7 

-0.2 

0.904 

0.0 

0.992 

Average  quarterly  employment  (%) 

62.5 

62.4 

64.2 

-1.7 

0.441 

-1.8 

0.346 

Had  employment  spell  of  at  least  4 quarters  (%) 

77.8 

75.3 

76.7 

1.0 

0.719 

-1.5 

0.548 

Advancement  measures 

Average  annual  earnings  ($) 

9,981 

9,348 

9,999 

-19 

0.974 

-651 

0.182 

Average  annual  earnings  of  $10,000  or  more  (%) 

41.6 

41.2 

41.4 

0.1 

0.971 

-0.2 

0.935 

Quarters  with  earnings  of  $3,500  or  more  (%) 

34.2 

30.8 

33.0 

1.2 

0.584 

-2.3 

0.229 

(continued) 
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Appendix  Table  B.17  (continued) 


Training 

Work 

Training  Focused 

Work  Plus 

Focused 

Plus 

Control 

Group 

Group 

Outcome 

Group 

Group 

Group 

Impact 

P-Value 

Impact 

P-Value 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

1,905 

1,945 

1,819 

86 

0.519 

126 

0.276 

Average  annual  food  stamps  received  ($) 

1,044 

1,134 

1,029 

15 

0.838 

105 

0.106 

Average  annual  income  ($) 

12,930 

12,427 

12,847 

83 

0.876 

-420 

0.358 

Sample  size  (total  = 1,746) 

898 

435 

413 

SOURCE:  MDRC  calculations  from  state  unemployment  insurance  (UI)  wage  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  Ei-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across  different  subgroups.  Statistical  significance 
levels  are  indicated  as  follows:  fff  = 1 percent;  ff  = 5 percent;  f = 10  percent. 


Appendix  C 

Supplementary  Exhibits  for  Chapter  5: 
Programs  Serving  Employed  Non-TANF  Recipients 

Appendix  C presents  supplementary  boxes  and  tables  relating  to  the  Cleveland,  Eugene  and 
Medford,  Riverside  PASS,  and  South  Carolina  ERA  tests. 

Box 

C.  1 Program  Summary:  Cleveland 

C.2  Program  Summary:  Eugene 

C.3  Program  Summary:  Medford 

C.4  Program  Summary:  Riverside  PASS 

C.5  Program  Summary:  South  Carolina 

Table 

C.  1 Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Cleveland 

C.2  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income:  Cleveland 

C.3  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-3):  Cleveland 

C.4  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Cleveland 

C.5  Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Eugene 

C.6  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income:  Eugene 
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Table 


C.7  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-3):  Eugene 

C.8  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Eugene 

C.9  Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Medford 

C.  10  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income:  Medford 

C.  1 1 Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-3):  Medford 

C.  12  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Medford 

C.  13  Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  Riverside  PASS 

C.  14  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Riverside  PASS 

C.  15  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4):  Riverside  PASS 

C.  16  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  Riverside  PASS 

C.  17  Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42 -Month  Survey 
for  the  Early  Cohort:  Riverside  PASS 

C.  18  Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42 -Month  Survey 
for  the  Later  Cohort:  Riverside  PASS 

C.  19  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4),  by  Type  of  Service  Provider:  Riverside  PASS 

C.20  Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42 -Month  Survey  for 
Participants  Served  by  a Community-Based  Organization:  Riverside  PASS 

C.21  Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey:  South  Carolina 

C.22  Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

South  Carolina 

C.23  Impacts  on  Employment  Retention  and  Advancement  in  the  Cumulative  Follow-Up 
Period  (Years  1-4):  South  Carolina 

C.24  Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey:  South  Carolina 

C.25  Impacts  on  Employment  Retention,  Advancement,  and  TANF  Receipt  in  the  Cumulative 
Follow-Up  Period  (Years  1-4),  by  Labor  Force  Attachment 
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Appendix  Box  C.l 


Program  Summary: 

Cleveland 

“Voluntary,  employer-based  retention  services  through  intensive  staff-client  engagement” 

Goal:  Increase  retention  among  low-wage,  entry-level  workers  in  the  long-term  nursing  care 
industry 

Location:  Cleveland,  Ohio 

Target  population:  Workers  earning  less  than  $13  per  hour,  hired  within  the  past  six  months, 
and  employed  by  42  long-term  nursing  care  firms  and  two  manufacturing  firms 

Implementation  schedule:  Eight  firms  were  randomly  assigned  in  September  2002,  followed 
by  36  firms  in  three  more  stages  — 14  firms  in  November  2002,  eight  firms  in  February  2004, 
and  14  firms  in  May  2004.  The  program  was  operated  between  September  2002  and  May  2005. 

Management  structure:  Operated  by  Towards  Employment,  a community-based  social  service 
organization 

Participation  requirements:  Participation  was  voluntary. 

Outreach  and  marketing:  Staff  heavily  marketed  services,  using  strategically  placed  flyers 
and  facilitywide  loudspeaker  announcements  encouraging  attendance  and  gamering  enthusiasm 
for  the  program. 

Staff-client  engagement: 

• Generalist  staff  (not  team-based)  were  stationed  at  the  employers’  sites  with  regularly 
scheduled  office  hours.  Staff  also  provided  activities  during  all  three  shifts  (day, 
evening,  and  night). 

• Staff  tailored  services  to  individual  participants. 

• Staff  increased  the  frequency  of  contacts  with  participants. 

Job  preparation  and  placement  services:  Not  a priority  of  the  model  and  not  pursued  by  staff 

Retention  services: 

• Lunch  and  Learn  sessions  — led  by  either  Cleveland  ERA  staff  or  one  of  Towards 
Employment’s  training  facilitators  — provided  life  skills  education. 

• Staff  provided  help  dealing  with  problems  that  might  put  individuals  at  risk  of  losing 
their  job,  such  as  transportation  or  child  care  needs. 

• Staff  provided  work  supports  and  social  service  referrals. 

Advancement  services:  Advancement  was  not  a priority  of  the  model  and  was  not  pursued  by 
staff. 

Employer  linkages:  Being  an  employer-based  program,  it  therefore  had  strong  connections 
with  participants’  employers. 


(continued) 
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Appendix  Box  C.l  (continued) 


Key  funding  source:  Public  and  private  grant  funding  raised  by  Towards  Employment 

Implementation  challenges: 

• Participation  challenges  due  to  the  round-the-clock  nature  of  duties  in  the  long-term 
nursing  care  industry 

• Difficulty  navigating  workplace  rules  and  space  constraints  in  the  delivery  of  services 
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Appendix  Box  C.2 


Program  Summary: 

Eugene 

“Individualized  assistance  and  career  counseling  for 
advancement,  through  welfare  and  community  college  partnership” 

Goal:  Help  former  TANF  recipients  retain  their  jobs  and  advance  to  better  jobs  and  wages 
Location:  Eugene,  Oregon 

Target  population:  Employed  individuals  who  had  left  TANF  within  the  year  and  who  were 
working  more  than  20  hours  per  week 

Implementation  schedule:  Random  assignment  occurred  from  June  2002  to  June  2004.  The 
program  operated  from  February  2002  to  July  2005. 

Management  structure:  Jointly  operated  by  the  Oregon  Department  of  Human  Services  (DHS) 
and  Lane  Community  College,  with  services  provided  in  the  DHS  offices 

Participation  requirements:  Participation  was  voluntary. 

Outreach  and  marketing:  Through  mail  and  phone,  staff  sought  to  schedule  intake  meetings, 
emphasizing  the  program’s  advancement  focus. 

Staff-client  engagement: 

• Staffed  by  two-person,  colocated  teams  that  provided  flexibility  when  accommodating 
participants. 

• Staff-client  interactions  were  highly  personalized  and  frequent. 

Job  preparation  and  placement  services:  Not  part  of  the  model  and  not  pursued  by  staff. 

Retention  services: 

• Job  loss  emerged  as  an  acute  issue  for  the  program.  Staff  provided  job  search 
assistance,  including  one-on-one  job  search  assistance,  help  with  creating  or  updating  a 
resume,  and  providing  job  leads. 

• Staff  provided  assistance  securing  food  stamps,  transitional  child  care,  and  subsidized 
health  insurance,  and  they  gave  participants  gas  vouchers  and  car  repair  money. 

• Staff  provided  social  service  referrals. 

• Job-specific  retention  services  were  informal  — usually  when  ERA  staff  discovered 
that  a participant  was  having  a problem  on  the  job. 

Advancement  services: 

• Staff  provided  referrals  to  education  programs,  particularly  for  vocational  and  skills 
training. 

• Staff  provided  job  leads  and  suggestions  for  job  advancement,  as  appropriate. 

• Staff  provided  goals-focused  career  counseling. 

Employer  linkages:  Not  part  of  the  model  design  and  not  pursued  by  staff. 

(continued) 
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Appendix  Box  C.2  (continued) 


Key  funding  source:  State  TANF  program.  Although  state  funding  was  reduced  during  the 
2001-2003  and  2003-2005  state  bienniums,  funds  from  the  Department  of  Labor  allowed 
services  to  continue. 

Implementation  challenges: 

• Cross-organizational  cultural  differences  hampered  working  relationships  and 
institutional  support. 

• High  caseloads  and  ERA  staff  policies  restricted  flexibility  for  participant  meetings. 

• Demands  for  reemployment  services  detracted  from  the  career  counseling  envisioned  in 
the  model. 
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Appendix  Box  C.3 


Program  Summary: 

Medford 

“Individualized  assistance  and  career  counseling  for  advancement, 
through  welfare  and  workforce  partnership” 

Goals:  Help  low- wage  working  people  retain  their  jobs  and  advance  to  better  jobs  and  wages 
Location:  Medford,  Oregon 

Target  population:  Employed  individuals  who  had  left  TANF  or  who  were  currently 
participating  in  the  Oregon  Food  Stamp  Employment  and  Training  program  or  the  Employment 
Related  Day  Care  program 

Implementation  schedule:  Random  assignment  occurred  from  February  2002  to  April  2004, 
and  the  program  operated  from  the  beginning  of  random  assignment  to  July  2005. 

Management  structure:  Jointly  operated  by  the  Oregon  Department  of  Human  Services,  The 
Job  Council,  the  Employment  Department,  and  Rogue  Community  College 

Participation  requirements:  Participation  was  voluntary. 

Outreach  and  marketing:  Staff  used  a proactive  approach  to  engagement. 

Staff-client  engagement: 

• Staff  teams  provided  group  members  with  flexibility  in  whom  they  met  with,  and 
frequent  contact  was  emphasized. 

• Services  were  driven  by  participants’  career  interests  and  participant-defined  goals. 

Job  preparation  and  placement  services:  Not  part  of  the  model  and  not  pursued  by  staff 

Retention  services: 

• Job  loss  emerged  as  an  acute  issue;  therefore,  staff  provided  assistance  in  preparing 
resumes  and  job  applications  and  identifying  job  leads. 

• Staff  provided  assistance  securing  food  stamps,  transitional  child  care,  and  subsidized 
health  insurance,  and  they  gave  participants  gas  vouchers  and  car  repair  money. 

• Staff  made  social  service  referrals  for  program  group  members  who  identified  needs. 

• Staff  spoke  with  participants  about  problems  that  they  were  having  on  the  job. 

Advancement  services: 

• Staff  and  participants  developed  and  used  Personal  Development  Plans. 

• Staff  assisted  participants  with  career  moves  or  asking  employers  for  raises  or 
promotions,  as  needed. 

• Staff  encouraged  training  and  education  as  an  advancement  strategy. 

Employer  linkages:  Originally  the  service  delivery  model  called  for  staff  to  reach  out  to  area 
employers  on  behalf  of  clients,  but  this  goal  never  fully  materialized  due  to  the  staffs  lack  of 
experience. 

(continued) 
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Appendix  Box  C.3  (continued) 


Key  funding  source:  State  TANF  program.  Although  state  funding  was  reduced  during  the 
2001-2003  and  2003-2005  state  bienniums,  funds  from  the  Department  of  Labor  allowed 
services  to  continue. 

Implementation  challenges: 

• The  program  was  beset  with  funding  difficulties  and  staff  turnover. 

• Reemployment  consumed  more  time  than  originally  anticipated. 

• Staff  initially  struggled  to  define  advancement  and  determine  how  to  help  clients 
progress. 


352 


Appendix  Box  C.4 


Program  Summary: 

Riverside  PASS  (Post-Assistance  Self-Sufficiency) 

“Intensive  and  multifaceted  postemployment  services  provided  by 
nonprofit  organizations  in  most  locations” 

Goal:  Job  retention  and  advancement  among  employed  individuals  who  recently  left  TANF 
Location:  Riverside  County,  California 

Target  population:  Employed  former  TANF  recipients  who  recently  left  TANF 

Implementation  schedule:  Random  assignment  occurred  from  July  2002  to  June  2003.  The 
program  operated  from  the  beginning  of  random  assignment  to  December  2003. 

Management  structure:  Operated  by  three  community-based  organizations,  a community 
college,  or  a Department  of  Public  Social  Services  office  — depending  on  clients’  residence. 

Participation  requirements:  Participation  was  voluntary. 

Outreach  and  marketing:  All  five  service  providers  made  a concerted,  sustained  effort  to 
contact  program  group  members  and  encourage  them  to  enroll  in  the  program. 

Staff-client  engagement: 

• The  program  used  various  service  delivery  models,  including  coordination  among 
partnering  organizations. 

• Staff  met  with  program  group  members  during  regular  work  hours  (rather  than  during 
night  and  weekend  shifts)  and  within  the  office. 

• Services  were  tailored  to  participants’  needs  and  interests. 

Job  preparation  and  placement  services:  Varied  by  provider  but  could  include  one-on-one 
job  search  assistance,  help  with  creating  or  updating  a resume,  or  providing  job  leads. 

Retention  services: 

• Job  loss  emerged  as  an  acute  issue  for  the  program.  As  a result,  staff  devoted 
considerable  attention  to  rapid  reemployment  services. 

• Staff  provided  assistance  payments  for  rent  and  utilities,  transportation  services 
(gasoline  vouchers  and  car  repair),  assessment  of  child  care  needs  and  help  with 
processing  payments,  food  assistance,  referrals  to  legal  services,  and  payments  for 
books  and  other  school  supplies  and  parking  fees. 

• Staff  made  social  service  referrals  for  program  group  members  who  identified  needs. 

Advancement  services:  Staff  provided  some  career  counseling  and  development  of 
employment  plans;  however,  such  services  were  varied  across  providers  and  were  largely 
informal.  Referrals  for  education  and  training  services  were  offered. 

Employer  linkages:  Not  included  in  the  model  and  not  pursued  by  staff 

(continued) 
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Appendix  Box  C.4  (continued) 


Key  funding  source:  State  TANF  funds 

Implementation  challenges: 

• Some  providers  were  inexperienced  at  running  postemployment  programs  for  former 
TANF  recipients. 

• Staff  had  difficulty  locating  sample  members  due  to  incomplete  or  out-of-date  contact 
information. 

• It  was  difficult  to  convince  participants  to  use  services. 
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Appendix  Box  C.5 


Program  Summary: 

South  Carolina 

“Pre-  and  postemployment  services  for  sustained  employment  and  advancement 

of  former  TANF  recipients” 

Goal:  Place  former  welfare  recipients  in  jobs,  help  them  sustain  employment  and  advance,  and 
increase  their  earnings 

Location:  Pee  Dee,  a region  that  includes  six  South  Carolina  Department  of  Social  Services 
(DSS)  offices  (in  Chesterfield,  Darlington,  Dillon,  Florence,  Marion,  and  Marlboro  Counties) 

Target  population:  Former  welfare  recipients  who  had  left  welfare  between  October  1997  and 
December  2000  and  had  not  returned  to  welfare 

Implementation  schedule:  Random  assignment  occurred  from  September  2001  to  January 
2003,  and  the  program  operated  from  the  beginning  of  random  assignment  to  April  2005. 

Management  structure:  Operated  by  each  of  the  six  county  DSS  offices  listed  above 

Participation  requirements:  Participation  was  voluntary. 

Outreach  and  marketing:  Staff  worked  hard  to  engage  participants,  after  receiving  marketing 
and  outreach  training,  and  offered  modest  participation  incentives  to  participants. 

Staff-client  engagement: 

• Generalist  staff  (not  team-based  structure)  were  available  — either  in  the  office  or  by 
phone  — beyond  the  standard  9 to  5 workday. 

• Services  were  tailored  to  individual  participants. 

• Staff  provided  career  assessments  and  developed  participant  employment  plans. 

Job  preparation  and  placement  services:  Unemployed  participants  received  assistance 
preparing  for  and  searching  for  a job. 

Retention  services: 

• Staff  provided  modest  financial  incentives  for  employment  achievements,  such  as 
retaining  a job  for  three  months  or  advancing  from  part-time  to  full-time  employment. 

• Staff  provided  assistance  with  securing  work  supports  and  help  getting  TANF,  food 
stamps,  and  Medicaid. 

• Staff  provided  social  service  referrals,  as  needed. 

• Staff  held  periodic  check-ins  and  talked  about  workplace  problems  and  concerns. 

Advancement  services: 

• Staff  provided  some  referrals  to  short-term  education  and  training. 

• Staff  strategized  with  some  participants  on  how  to  ask  for  a raise  or  how  to  learn  about 
promotion  opportunities. 

Employer  linkages:  No  formal  connection  between  the  program  and  local  employers 

(continued) 
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Appendix  Box  C.5  (continued) 


Key  funding  source:  State  TANF  funds 

Implementation  challenges: 

• Some  staff  lacked  experience  working  with  employed  former  TANF  recipients. 

• Funding  interruptions  presented  operations  challenges. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.l 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Cleveland 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program3  (%) 

52.4 

30.2 

22.2  *** 

0.000 

Average  number  of  contacts  with  staff 

6.8 

3.1 

^ g *** 

0.007 

Talked  with  staff  in  past  4 weeks  (%) 

16.6 

8.4 

8.2  ** 

0.035 

Staff  ever  talked  with  respondent's  employer  (%) 

13.9 

7.5 

6.4  ** 

0.048 

Services 

Received  help  with  support  services  (%) 

19.5 

16.6 

2.9 

0.389 

Received  help  with  basic  needs  (%) 

24.5 

28.1 

-3.6 

0.395 

Received  help  with  public  benefits  (%) 

41.0 

39.8 

1.2 

0.791 

Received  help  with  job  preparation  (%) 

18.5 

13.7 

4.8 

0.187 

Received  help  with  retention/advancement  (%) 

38.5 

27.5 

11.0  ** 

0.025 

Finding  a better  job  while  working 

6.3 

3.9 

2.4 

0.330 

Other  activities  while  working 

18.4 

1.9 

16.5  *** 

0.000 

Career  assessment 

9.0 

5.6 

3.5 

0.178 

Dealing  with  problems  on  the  job 

17.9 

7.7 

10.2  *** 

0.005 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

11.3 

6.5 

4.9  * 

0.135 

Participation 

Participated  in  any  activity  (%) 

61.0 

58.8 

2.2 

0.643 

Participated  in  any  employment-related  activity3  (%) 

45.2 

35.3 

9 9 ** 

0.103 

Participated  in  a job  search  activity  (%) 

44.6 

34.2 

10.4  ** 

0.086 

Group  job  search/job  club 

25.7 

13.0 

0.004 

Individual  job  search 

30.5 

29.6 

0.9 

0.853 

Participated  in  an  education/training  activity  (%) 

28.9 

37.3 

-8.3  * 

0.152 

ABE/GED  and  ESL 

12.3 

8.0 

4.3 

0.262 

College  courses 

15.3 

25.1 

_9  g *** 

0.031 

Vocational  training 

8.9 

11.9 

-3.0 

0.547 

Participated  in  unpaid  work/subsidized  employment  (%) 

2.5 

3.8 

-1.3 

0.454 

Participated  in  an  education  activity  while  working  (%) 

22.7 

29.8 

-7.2  * 

0.232 

Participated  in  an  employment  activitiy  while  working  (%) 

14.2 

12.9 

1.4 

0.643 

Average  number  of  weeks  participating  in 

Job  search  activities 

5.8 

3.5 

73** 

0.112 

Education/training  activities 

5.6 

6.9 

-1.3 

0.362 

Unpaid  work/subsidized  employment 

0.4 

0.5 

-0.1 

0.780 

Sample  size  (total  = 485) 

260 

225 

(continued) 
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Appendix  Table  C.l  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.2 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Cleveland 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Standard 

Error 

FoIIow-Ud  Year  1 

Ever  employed  (%) 

98.8 

97.4 

1.4 

0.192 

1.0 

Average  quarterly  employment  (%) 

90.6 

87.7 

2.8 

0.308 

1.7 

Employed  every  quarter  (%) 

78.4 

71.6 

6.9 

0.214 

3.4 

Employed  with  annual  earnings  over  $10,000  (%) 

73.6 

72.0 

1.7 

0.654 

3.4 

Annual  earnings  ($) 

15,335 

15,018 

317 

0.637 

569 

FoIIow-Ud  Year  2 

Ever  employed  (%) 

90.4 

91.9 

-1.5 

0.474 

2.2 

Average  quarterly  employment  (%) 

81.9 

82.1 

-0.2 

0.861 

2.5 

Employed  every  quarter  (%) 

70.0 

69.4 

0.6 

0.914 

3.6 

Employed  with  annual  earnings  over  $10,000  (%) 

65.0 

64.0 

1.1 

0.798 

3.7 

Annual  earnings  ($) 

14,273 

14,536 

-263 

0.709 

719 

FoIIow-Ud  Year  3 

Ever  employed  (%) 

86.2 

86.3 

-0.1 

0.979 

2.7 

Average  quarterly  employment  (%) 

77.4 

76.0 

1.4 

0.659 

2.8 

Employed  every  quarter  (%) 

65.8 

63.1 

2.7 

0.482 

3.6 

Employed  with  annual  earnings  over  $10,000  (%) 

60.9 

58.8 

2.1 

0.648 

3.7 

Annual  earnings  ($) 

13,777 

14,146 

-368 

0.642 

781 

Sample  size  (total  = 697) 

381 

316 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups 
arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 
The  outcome  measures  are  discussed  in  Chapter  2. 


359 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.3 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-3): 

Cleveland 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  measures 

Number  of  quarters  employed  at  jobs  held 

in  the  quarter  after  random  assignment* 

5.4 

5.2 

0.3 

0.429 

Average  quarterly  employment  (%) 

83.3 

81.9 

1.3 

0.664 

Percentage  of  quarters  employed  (%) 

Never  employed 

0.9 

1.8 

-0.9 

0.296 

1-25 

6.2 

5.5 

0.7 

0.652 

26-50 

8.7 

7.0 

1.7 

0.429 

51-75 

11.2 

16.5 

-5.3  * 

0.064 

76-100 

73.0 

69.2 

3.8 

0.289 

Employed  entire  follow-up  period  (%) 

52.9 

47.4 

5.5 

0.167 

Had  employment  spell  of  at  least  4 quarters  (%) 

88.9 

89.7 

-0.8 

0.712 

Average  number  of  employers  during  follow-up  period 

4.1 

4.1 

0.0 

0.900 

Number  of  employers  (%) 

Never  employed 

0.9 

1.8 

-0.9 

0.296 

1 to  2 

37.6 

33.8 

3.8 

0.395 

3 to  4 

26.5 

27.8 

-1.3 

0.791 

5 to  8 

24.7 

27.5 

-2.7 

0.443 

More  than  8 

10.3 

9.1 

1.1 

0.736 

Average  number  of  quarters  in  first  employment  spell 

8.6 

8.4 

0.2 

0.621 

Number  of  quarters  until  first  employment  spell 

0.2 

0.4 

-0.2  * 

0.071 

Average  number  of  employment  spells 

1.4 

1.4 

0.0 

0.823 

Average  length,  longest  employment  spell  (quarters) 

9.1 

9.0 

0.1 

0.849 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

0.9 

1.8 

-0.9 

0.296 

1 to  2 

6.6 

4.4 

2.3 

0.234 

3 to  4 

8.9 

7.6 

1.3 

0.564 

5 to  8 

19.1 

25.4 

-6.3  * 

0.103 

9 to  12 

64.5 

60.9 

3.6 

0.446 

(continued) 
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Appendix  Table  C.3  (continued) 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Average  number  of  unemployment  spells 

0.6 

0.7 

-0.1 

0.288 

Length  of  longest  unemployment  spell,  in  quarters 

1.8 

1.9 

-0.2 

0.689 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

52.9 

47 .4 

5.5 

0.167 

1 to  2 

23.6 

28.8 

-5.1 

0.150 

3 to  4 

7.3 

9.3 

-2.0 

0.506 

5 to  8 

10.3 

8.2 

2.1 

0.436 

9 to  12 

6.0 

6.4 

-0.5 

0.959 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

Average  annual  earnings  (%) 

14,462 

14,566 

-105 

0.858 

$0 

0.9 

1.8 

-0.9 

0.296 

$141,999 

5.8 

5.4 

0.4 

0.778 

$2,00044,999 

8.2 

8.4 

-0.2 

0.934 

$5,00049,999 

16.0 

19.0 

-3.0 

0.415 

$10,000414,999 

24.1 

18.7 

5.4  * 

0.132 

$15,000419,999 

21.6 

19.3 

2.3 

0.486 

$20,000  or  higher 

23.5 

27.4 

-4.0 

0.219 

Average  annual  earnings  of  $ 1 0,000  or  more  (%) 

69.1 

65.4 

3.7 

0.410 

Quarters  with  earnings  of  $3,500  or  more  (%) 

54.6 

54.3 

0.3 

0.920 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

0.9 

1.8 

-0.9 

0.296 

No  quarters  with  earnings  above  $3,500 

9.4 

11.1 

-1.7 

0.634 

1 to  2 

12.9 

10.7 

2.3 

0.395 

3 to  4 

13.6 

13.4 

0.2 

0.948 

5 to  8 

24.9 

23.5 

1.5 

0.675 

9 to  12 

38.3 

39.6 

-1.3 

0.731 

Comparison  of  auarter  with  highest  earnings  in  Year  1 
and  auarter  with  highest  earnings  in  Year  3 

Not  employed  in  either  Year  1 or  Year  3 

0.9 

2.1 

-1.3 

0.182 

Earnings  decreased  (%) 

No  longer  employed 

12.9 

11.6 

1.3 

0.612 

Earnings  decreased  by  less  than  $250 

4.4 

3.2 

1.2 

0.570 

Earnings  decreased  by  $250  or  more 

35.3 

36.5 

-1.2 

0.752 

Earnings  increased  (%) 

Became  employed 

0.4 

0.5 

-0.1 

0.837 

Earnings  increased  by  less  than  $250 

5.0 

3.8 

1.1 

0.510 

Earnings  increased  by  $250  or  more 

40.9 

41.5 

-0.6 

0.873 

Average  earnings  per  quarter  employed  ($) 

4,342 

4.445 

-103 

Sample  size  (total  = 697) 

381 

316 

(continued) 
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Appendix  Table  C.3  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given  to  those  who  were 
not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.4 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Cleveland 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

90.0 

90.2 

-0.2 

0.956 

Currently  employed 

71.7 

77.6 

-6.0 

0.188 

No  longer  employed 

18.4 

12.6 

5.8  * 

0.105 

Current  working  status  (%) 

Full  time 

64.1 

64.1 

0.0 

0.996 

Part  time 

7.1 

13.6 

-6.5  ** 

0.039 

Currently  employed  at  a "good  job"a  (%) 

50.0 

49.8 

0.1 

0.976 

Hours 

Average  hours  per  week 

27.1 

28.8 

-1.7 

0.344 

Total  hours  per  week  (%) 

Less  than  30 

7.1 

13.6 

-6.5  ** 

0.039 

30-34 

6.6 

8.4 

-1.8 

0.485 

35-44 

51.1 

46.3 

4.9 

0.379 

45  or  more 

6.4 

9.5 

-3.0 

0.262 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.0 

0.7 

0.3 

0.767 

$5.00  - $6.99 

4.7 

5.7 

-1.0 

0.654 

$7.00  - $8.99 

20.7 

20.1 

0.7 

0.865 

$9.00  or  more 

44.8 

51.3 

-6.5 

0.175 

Average  weekly  earnings  ($) 

259 

273 

-15 

0.409 

Total  earnings  per  week  (%) 

Less  than  $200 

6.2 

8.0 

-1.8 

0.486 

$201-$300 

11.7 

15.8 

-4.1 

0.277 

$301-$500 

46.9 

48.4 

-1.5 

0.752 

$500  or  more 

6.4 

5.5 

0.9 

0.712 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  fall  pay 

47.5 

46.0 

1.5 

0.748 

Paid  vacation 

57.0 

57.7 

-0.7 

0.877 

Paid  holidays  other  than  Christmas  and  New  Year 

57.0 

53.7 

3.3 

0.483 

Dental  benefits 

48.6 

49.6 

-1.0 

0.791 

A retirement  plan 

45.1 

44.4 

0.7 

0.912 

A health  plan  or  medical  insurance 

54.0 

55.9 

-1.9 

0.662 

(continued) 
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Appendix  Table  C.4  (continued) 


ERA  Control  Difference 

Outcome Group  Group  (Impact) P -Value 

Schedule"  (%) 


Regular 

35.4 

33.3 

2.1 

0.744 

Split  shift 

1.6 

1.8 

-0.2 

0.883 

Irregular 

1.2 

3.9 

-2.7  * 

0.094 

Evening  shift 

17.1 

17.6 

-0.5 

0.893 

Night  shift 

9.1 

14.0 

-4.9 

0.119 

Rotating  shift 

5.5 

6.5 

-1.0 

0.671 

Other  schedule 

1.5 

0.5 

1.0 

0.332 

Odd  job 

0.0 

0.0 

0.0 

0.332 

Job  skills 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

58.7 

62.9 

-4.2 

0.392 

Work  with  computers 

17.3 

17.0 

0.3 

0.864 

Arithmetic  skills 

25.2 

29.2 

-4.0 

0.393 

Customer  contact 

66.3 

72.7 

-6.4 

0.193 

Sample  size  (total  = 485) 

260 

225 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 

by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  *** 

= 1 percent;  ** 

= 5 percent; 

* = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.5 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 


Eugene 


ERA  Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P -Value 

Contacts 

Ever  had  contact  with  staff/ employment  program  (%) 

83.5 

85.6 

-2.2 

0.537 

Average  number  of  contacts  with  staff 

21.3 

17.6 

3.7 

0.142 

Talked  with  staff  in  past  4 weeks  (%) 

44.3 

39.4 

4.9 

0.311 

Staff  ever  talked  with  respondent's  employer  (%) 

10.4 

9.6 

0.8 

0.787 

Services 

Received  help  with  support  services  (%) 

65.8 

58.8 

7.0 

0.124 

Received  help  with  basic  needs  (%) 

58.2 

49.6 

8.6  * 

0.076 

Received  help  with  public  benefits  (%) 

79.8 

73.8 

6.0 

0.142 

Received  help  with  job  preparation  (%) 

49.9 

40.5 

9.4  * 

0.053 

Received  help  with  retention/advancement  (%) 

38.0 

22.0 

16.0  *** 

0.000 

Finding  a better  job  while  working 

15.5 

6.8 

g g *** 

0.004 

Other  activities  while  working 

17.9 

6.7 

j 1 o *** 

0.000 

Career  assessment 

17.8 

9.5 

8.3  ** 

0.014 

Dealing  with  problems  on  the  job 

9.6 

5.9 

3.6 

0.158 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

11.4 

9.1 

2.4 

0.421 

ParticiDation 

Participated  in  any  activity  (%) 

80.9 

74.1 

* 

oo 

SO 

0.094 

Participated  in  any  employment-related  activity2  (%) 

75.0 

68.7 

6.3 

0.141 

Participated  in  a job  search  activity  (%) 

74.5 

67.3 

7.3  * 

0.094 

Group  job  search/job  club 

56.1 

49.8 

6.3 

0.182 

Individual  job  search 

68.3 

62.6 

5.7 

0.207 

Participated  in  an  education/training  activity  (%) 

21.7 

25.1 

-3.4 

0.416 

ABE/GED  and  ESL 

4.2 

5.8 

-1.6 

0.439 

College  courses 

13.8 

18.9 

-5.1 

0.154 

Vocational  training 

5.1 

1.7 

3.4  * 

0.054 

Participated  in  unpaid  work/ subsidized  employment  (%) 

17.7 

20.0 

-2.3 

0.539 

Participated  in  an  education  activity  while  working  (%) 

14.0 

14.2 

-0.2 

0.942 

Participated  in  an  employment  activitiy  while  working  (%) 

27.8 

20.9 

6.9  * 

0.091 

Average  number  of  weeks  participating  in 

Job  search  activities 

11.1 

10.8 

0.3 

0.848 

Education/ training  activities 

3.8 

4.9 

-1.2 

0.300 

Unpaid  work/subsidized  employment 

2.1 

2.8 

-0.7 

0.344 

Sample  size  (total  = 440) 

220 

220 

(continued) 
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Appendix  Table  C.5  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.6 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 


Eugene 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Standard 

Error 

Fo11ow-Ud  Year  1 

Ever  employed  (%) 

91.4 

88.9 

2.5 

0.151 

1.7 

Average  quarterly  employment  (%) 

71.0 

69.0 

2.1 

0.300 

2.0 

Employed  every  quarter  (%) 

49.9 

48.2 

1.8 

0.535 

2.8 

Employed  with  annual  earnings  over  $ 10,000  (%) 

39.0 

37.4 

1.7 

0.543 

2.7 

Annual  earnings  ($) 

8,406 

8,365 

40 

0.919 

398 

Ever  received  TANF  (%) 

34.7 

38.6 

-3.9 

0.147 

2.7 

Ever  received  food  stamps  (%) 

95.3 

96.1 

-0.8 

0.523 

1.2 

Total  income  ($) 

11,733 

11,779 

-46 

0.901 

366 

Fo11ow-Ud  Year  2 

Ever  employed  (%) 

76.5 

76.6 

-0.1 

0.966 

2.4 

Average  quarterly  employment  (%) 

61.3 

61.4 

-0.1 

0.969 

2.3 

Employed  every  quarter  (%) 

42.9 

45.5 

-2.7 

0.341 

2.8 

Employed  with  annual  earnings  over  $ 10,000  (%) 

37.4 

37.9 

-0.4 

0.874 

2.7 

Annual  earnings  ($) 

8,479 

8,303 

176 

0.710 

474 

Ever  received  TANF  (%) 

22.2 

22.4 

-0.2 

0.934 

2.4 

Ever  received  food  stamps  (%) 

82.6 

81.6 

1.1 

0.633 

2.2 

Total  income  ($) 

11,249 

11,170 

79 

0.861 

453 

Fo11ow-Ud  Year  3 

Ever  employed  (%) 

65.1 

72.1 

_~j  o *** 

0.009 

2.7 

Average  quarterly  employment  (%) 

51.3 

57.8 

5 *** 

0.008 

2.4 

Employed  every  quarter  (%) 

35.9 

42.6 

-6.6  ** 

0.017 

2.8 

Employed  with  annual  earnings  over  $ 10,000  (%) 

34.1 

35.8 

-1.7 

0.532 

2.7 

Annual  earnings  ($) 

7,689 

8,543 

-854 

0.104 

526 

Ever  received  TANF  (%) 

19.9 

19.1 

0.8 

0.718 

2.3 

Ever  received  food  stamps  (%) 

74.5 

74.1 

0.4 

0.884 

2.5 

Total  income  ($) 

10,322 

11,042 

-721 

0.156 

508 

Sample  size  (total  = 1,179) 

585 

594 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.7 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-3): 

Eugene 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held 

in  the  quarter  after  random  assignment3 

3.3 

3.4 

-0.1 

0.791 

Average  quarterly  employment  (%) 

61.2 

62.7 

-1.5 

0.431 

Percentage  of  quarters  employed  (%) 

Never  employed 

4.1 

6.3 

-2.2  * 

0.086 

1-25 

18.7 

16.2 

2.5 

0.260 

26-50 

17.7 

16.7 

1.0 

0.659 

51-75 

19.5 

17.5 

2.0 

0.372 

76-100 

40.0 

43.3 

-3.3 

0.236 

Employed  entire  follow-up  period  (%) 

23.3 

27.9 

l 

4^ 

* 

0.060 

Had  employment  spell  of  at  least  4 quarters  (%) 

66.9 

68.8 

-1.9 

0.481 

Average  number  of  employers  during  follow-up  period 

2.9 

2.9 

0.0 

0.739 

Number  of  employers  (%) 

Never  employed 

4.1 

6.3 

-2.2  * 

0.086 

1 to  2 

47.2 

45.6 

1.6 

0.598 

3 to  4 

28.4 

28.0 

0.4 

0.888 

5 to  8 

18.0 

18.3 

-0.2 

0.916 

More  than  8 

2.4 

1.9 

0.5 

0.526 

Average  number  of  quarters  in  first  employment  spell 

5.8 

5.9 

-0.1 

0.630 

Number  of  quarters  until  first  employment  spell 

0.9 

1.2 

-0.3  * 

0.094 

Average  number  of  employment  spells 

1.5 

1.5 

0.0 

0.913 

Average  length,  longest  employment  spell  (quarters) 

6.4 

6.6 

-0.2 

0.401 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

4.1 

6.3 

-2.2  * 

0.086 

1 to  2 

19.1 

16.9 

2.1 

0.340 

3 to  4 

16.6 

15.0 

1.6 

0.466 

5 to  8 

25.1 

23.4 

1.6 

0.515 

9 to  12 

35.2 

38.4 

-3.1 

0.254 

(continued) 
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Appendix  Table  C.7  (continued) 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Average  number  of  unemployment  spells 

1.2 

1.1 

0.1 

0.293 

Length  of  longest  unemployment  spell,  in  quarters 

4.0 

3.8 

0.2 

0.396 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

23.3 

27.9 

-4.7  * 

0.060 

1 to  2 

24.0 

23.3 

0.7 

0.767 

3 to  4 

15.8 

13.6 

2.2 

0.282 

5 to  8 

20.3 

18.4 

2.0 

0.388 

9 to  12 

16.6 

16.9 

-0.3 

0.882 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

Average  annual  earnings  (%) 

8,191 

8,404 

-213 

0.601 

$0 

4.1 

6.3 

-2.2  * 

0.086 

$l-$  1,999 

21.9 

19.5 

2.5 

0.292 

$2,000- $4,999 

19.0 

18.2 

0.8 

0.720 

$5,000- $9, 999 

19.3 

20.7 

-1.4 

0.545 

$10,000-$  14,999 

17.5 

16.9 

0.6 

0.804 

$15,000-$19,999 

10.9 

10.8 

0.2 

0.930 

$20,000  or  higher 

7.3 

7.6 

-0.4 

0.809 

Average  annual  earnings  of  $10,000  or  more  (%) 

35.7 

35.4 

0.4 

0.891 

Quarters  with  earnings  of  $3,500  or  more  (%) 

27.3 

27.0 

0.3 

0.882 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

4.1 

6.3 

-2.2  * 

0.086 

No  quarters  with  earnings  above  $3,500 

37.9 

34.7 

3.2 

0.245 

1 to  2 

16.5 

17.8 

-1.3 

0.553 

3 to  4 

9.5 

9.9 

-0.4 

0.824 

5 to  8 

15.7 

15.5 

0.2 

0.939 

9 to  12 

16.4 

15.8 

0.6 

0.773 

Comnarison  of  auarter  with  highest  earnings  in  Year  1 
and  auarter  with  highest  earnings  in  Year  3 

Not  employed  in  either  Year  1 or  Year  3 

4.9 

7.0 

-2.1 

0.120 

Earnings  decreased  (%) 

No  longer  employed 

30.1 

20.9 

q 9 *** 

0.000 

Earnings  decreased  by  less  than  $250 

3.1 

3.0 

0.0 

0.967 

Earnings  decreased  by  $250  or  more 

19.9 

23.9 

-4.0 

0.103 

Earnings  increased  (%) 

Became  employed 

3.7 

4.1 

-0.3 

0.764 

Earnings  increased  by  less  than  $250 

3.8 

3.9 

-0.1 

0.910 

Earnings  increased  by  $250  or  more 

34.5 

37.1 

-2.6 

0.352 

Average  earnings  per  quarter  employed  ($) 

3,344 

3,350 

-5 

Sample  size  (total  = 1,179) 

585 

594 

(continued) 
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Appendix  Table  C.7  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given  to  those  who  were 
not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.8 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Eugene 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

EmDlovment  status 

Ever  employed  since  random  assignment  (%) 

94.3 

92.5 

1.9 

0.436 

Currently  employed 

65.3 

68.8 

-3.5 

0.440 

No  longer  employed 

29.1 

23.7 

5.4 

0.210 

Current  working  status  (%) 

Full  time 

49.1 

49.6 

-0.5 

0.912 

Part  time 

16.2 

19.2 

-3.0 

0.425 

Currently  employed  at  a "good  job"3  (%) 

28.4 

28.0 

0.5 

0.913 

Hours 

Average  hours  per  week 

22.0 

22.8 

-0.8 

0.642 

Total  hours  per  week  (%) 

Less  than  30 

16.2 

19.2 

-3.0 

0.425 

30-34 

11.7 

13.3 

-1.7 

0.606 

35-44 

30.8 

28.3 

2.6 

0.561 

45  or  more 

5.6 

8.1 

-2.5 

0.290 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.4 

1.8 

-0.3 

0.784 

$5.00  - $6.99 

2.7 

5.5 

-2.8 

0.141 

$7.00  - $8.99 

41.8 

33.2 

8.5  * 

0.073 

$9.00  or  more 

19.4 

28.3 

-8.9  ** 

0.027 

Average  weekly  earnings  ($) 

194 

209 

-15 

0.368 

Total  earnings  per  week  (%) 

Less  than  $200 

13.1 

16.0 

-3.0 

0.393 

$201 -$300 

23.8 

19.4 

4.4 

0.266 

$301-$500 

24.3 

26.6 

-2.3 

0.586 

$500  or  more 

4.1 

6.8 

-2.7 

0.213 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

17.8 

22.3 

-4.5 

0.241 

Paid  vacation 

29.8 

32.5 

-2.7 

0.538 

Paid  holidays  other  than  Christmas  and  New  Year 

28.5 

28.4 

0.1 

0.977 

Dental  benefits 

23.7 

28.6 

-5.0 

0.244 

A retirement  plan 

19.3 

27.5 

-8.2  ** 

0.045 

A health  plan  or  medical  insurance 

29.2 

35.4 

-6.2 

0.171 

(continued) 
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Appendix  Table  C.8  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Schedule1"  (%) 

Regular 

39.0 

42.4 

-3.4 

0.478 

Split  shift 

1.5 

0.3 

1.1 

0.223 

Irregular 

4.6 

5.4 

-0.8 

0.690 

Evening  shift 

8.3 

9.0 

-0.8 

0.780 

Night  shift 

4.8 

3.0 

1.8 

0.340 

Rotating  shift 

4.4 

5.6 

-1.1 

0.601 

Other  schedule 

2.3 

0.5 

1.8 

0.111 

Odd  job 

0.5 

2.2 

-1.7 

0.135 

Job  skills 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

52.6 

58.3 

-5.6 

0.242 

Work  with  computers 

26.7 

31.1 

-4.4 

0.316 

Arithmetic  skills 

39.8 

43.3 

-3.5 

0.470 

Customer  contact 

57.2 

57.8 

-0.5 

0.915 

Sample  size  (total  = 440) 

220 

220 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.9 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Medford 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

73.1 

62.3 

10.8  ** 

0.039 

Average  number  of  contacts  with  staff 

10.1 

6.4 

3.8  ** 

0.023 

Talked  with  staff  in  past  4 weeks  (%) 

24.2 

24.5 

-0.3 

0.953 

Staff  ever  talked  with  respondent's  employer  (%) 

9.5 

7.3 

2.2 

0.475 

Services 

Received  help  with  support  services  (%) 

38.8 

38.9 

-0.1 

0.981 

Received  help  with  basic  needs  (%) 

40.5 

37.3 

3.2 

0.561 

Received  help  with  public  benefits  (%) 

55.2 

66.2 

-10.9  ** 

0.042 

Received  help  with  job  preparation  (%) 

26.3 

19.2 

7.1 

0.115 

Received  help  with  retention/advancement  (%) 

24.8 

16.1 

* 

* 

OO 

o 6 

0.049 

Finding  a better  job  while  working 

10.9 

3.3 

7 6 *** 

0.007 

Other  activities  while  working 

10.3 

2.7 

7 7 *** 

0.005 

Career  assessment 

16.5 

7.0 

9 6 *** 

0.006 

Dealing  with  problems  on  the  job 

6.0 

3.9 

2.2 

0.372 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

4.4 

6.0 

-1.6 

0.518 

ParticiDation 

Participated  in  any  activity  (%) 

63.4 

55.7 

7.6 

0.163 

Participated  in  any  employment-related  activity3  (%) 

48.1 

43.7 

4.4 

0.416 

Participated  in  a job  search  activity  (%) 

46.4 

43.0 

3.4 

0.532 

Group  job  search/job  club 

25.3 

20.6 

4.7 

0.315 

Individual  job  search 

39.8 

39.7 

0.1 

0.987 

Participated  in  an  education/training  activity  (%) 

30.3 

21.6 

8.7  * 

0.074 

ABE/GED  and  ESL 

6.1 

3.9 

2.2 

0.325 

College  courses 

20.5 

13.3 

7.2  * 

0.084 

Vocational  training 

6.4 

5.3 

1.1 

0.669 

Participated  in  unpaid  work/subsidized  employment  (%) 

8.7 

7.6 

1.0 

0.722 

Participated  in  an  education  activity  while  working  (%) 

28.0 

15.3 

Y2  7 *** 

0.005 

Participated  in  an  employment  activity  while  working  (%) 

21.5 

13.5 

8.0  * 

0.059 

Average  number  of  weeks  participating  in 

Job  search  activities 

4.9 

5.1 

-0.2 

0.853 

Education/training  activities 

6.5 

4.4 

2.1 

0.147 

Unpaid  work/subsidized  employment 

1.7 

1.7 

0.0 

0.980 

Sample  size  ( total  = 345) 

167 

178 

(continued) 
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Appendix  Table  C.9  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 


Appendix  Table  C.10 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Medford 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Standard 

Error 

Folio w-Ud  Year  1 

Ever  employed  (%) 

90.4 

93.1 

-2.7  * 

0.083 

1.6 

Average  quarterly  employment  (%) 

76.7 

79.4 

-2.7 

0.143 

1.8 

Employed  every  quarter  (%) 

59.6 

62.8 

-3.2 

0.247 

2.7 

Employed  with  annual  earnings  over  $10,000  (%) 

44.0 

49.5 

-5  5 ** 

0.042 

2.7 

Annual  earnings  ($) 

9,935 

10,351 

-416 

0.270 

377 

Ever  received  TANF  (%) 

9.6 

8.7 

0.9 

0.586 

1.7 

Ever  received  food  stamps  (%) 

88.3 

91.3 

-2.9  * 

0.086 

1.7 

Total  income  ($) 

12,228 

12,665 

-436 

0.222 

357 

Folio  w-Ud  Year  2 

Ever  employed  (%) 

83.3 

85.2 

-1.9 

0.363 

2.1 

Average  quarterly  employment  (%) 

70.5 

71.3 

-0.8 

0.699 

2.2 

Employed  every  quarter  (%) 

55.3 

55.2 

0.1 

0.986 

2.9 

Employed  with  annual  earnings  over  $10,000  (%) 

45.3 

47.9 

-2.6 

0.348 

2.8 

Annual  earnings  ($) 

9,763 

10,259 

-496 

0.285 

464 

Ever  received  TANF  (%) 

8.5 

9.1 

-0.6 

0.709 

1.6 

Ever  received  food  stamps  (%) 

78.2 

78.3 

-0.1 

0.979 

2.3 

Total  income  ($) 

11,856 

12,338 

-482 

0.278 

444 

Folio  w-Ud  Year  3 

Ever  employed  (%) 

79.6 

80.0 

-0.4 

0.853 

2.3 

Average  quarterly  employment  (%) 

67.4 

67.8 

-0.3 

0.881 

2.3 

Employed  every  quarter  (%) 

53.8 

54.0 

-0.2 

0.952 

2.9 

Employed  with  annual  earnings  over  $10,000  (%) 

45.1 

49.5 

-4.5 

0.117 

2.9 

Annual  earnings  ($) 

10,333 

10,851 

-517 

0.336 

538 

Ever  received  TANF  (%) 

11.0 

7.9 

3.2  * 

0.062 

1.7 

Ever  received  food  stamps  (%) 

70.1 

63.5 

6.7  ** 

0.013 

2.7 

Total  income  ($) 

12,312 

12,670 

-359 

0.489 

519 

Sample  size  (total  =1,1 64) 

590 

574 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.ll 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-3): 

Medford 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Emolovment  measures 

Number  of  quarters  employed  at  jobs  held 

in  the  quarter  after  random  assignment3 

4.1 

4.6 

-0.5  ** 

0.049 

Average  quarterly  employment  (%) 

71.5 

72.8 

-1.3 

0.457 

Percentage  of  quarters  employed  (%) 

Never  employed 

5.2 

2.1 

3 j *** 

0.005 

1-25 

10.6 

11.0 

-0.4 

0.812 

26-50 

10.7 

12.7 

-2.0 

0.291 

51-75 

17.5 

19.4 

-1.9 

0.400 

76-100 

56.0 

54.7 

1.3 

0.653 

Employed  entire  follow-up  period  (%) 

33.9 

36.4 

-2.5 

0.356 

Had  employment  spell  of  at  least  4 quarters  (%) 

80.4 

80.1 

0.3 

0.885 

Average  number  of  employers  during  follow-up  period 

3.3 

3.0 

0.2  * 

0.085 

Number  of  employers  (%) 

Never  employed 

5.2 

2.1 

3 j *** 

0.005 

1 to  2 

40.7 

45.6 

-4.9  * 

0.089 

3 to  4 

29.2 

33.1 

-3.9 

0.155 

5 to  8 

21.3 

16.6 

4.8  ** 

0.038 

More  than  8 

3.6 

2.6 

1.0 

0.339 

Average  number  of  quarters  in  first  employment  spell 

6.8 

7.2 

-0.3 

0.203 

Number  of  quarters  until  first  employment  spell 

1.0 

0.7 

0.4  ** 

0.012 

Average  number  of  employment  spells 

1.4 

1.5 

-0.1 

0.192 

Average  length,  longest  employment  spell  (quarters) 

7.6 

7.8 

-0.1 

0.532 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

5.2 

2.1 

3 j *** 

0.005 

1 to  2 

9.1 

11.6 

-2.5 

0.169 

3 to  4 

13.2 

13.6 

-0.3 

0.866 

5 to  8 

26.2 

24.8 

1.4 

0.597 

9 to  12 

46.3 

47.9 

-1.7 

0.560 

(continued) 
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Appendix  Table  C.ll  (continued) 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Average  number  of  unemployment  spells 

0.9 

1.0 

0.0 

0.690 

Length  of  longest  unemployment  spell,  in  quarters 

3.0 

2.8 

0.2 

0.249 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

33.9 

36.4 

-2.5 

0.356 

1 to  2 

26.4 

25.1 

1.3 

0.617 

3 to  4 

13.9 

13.1 

0.8 

0.706 

5 to  8 

13.7 

15.6 

-1.9 

0.352 

9 to  12 

12.2 

9.8 

2.4 

0.187 

Earninss  and  advancement  measures 

Average  annual  earnings  ($) 

10,010 

10,487 

-476 

0.225 

Average  annual  earnings  (%) 

$0 

5.2 

2.1 

3 1 *** 

0.005 

$l-$  1,999 

10.2 

12.5 

-2.3 

0.214 

$2,000-$4,999 

15.4 

14.5 

0.9 

0.652 

$5,000-$9,999 

24.6 

23.4 

1.2 

0.635 

$10,000-$  14,999 

20.3 

20.2 

0.1 

0.965 

$15,000-$  19,999 

14.1 

14.7 

-0.6 

0.765 

$20,000  or  higher 

10.2 

12.6 

-2.4 

0.166 

Average  annual  earnings  of  $10,000  or  more  (%) 

44.6 

47.5 

-2.9 

0.291 

Quarters  with  earnings  of  $3,500  or  more  (%) 

35.2 

36.7 

-1.4 

0.432 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

5.2 

2.1 

3 Y *** 

0.005 

No  quarters  with  earnings  above  $3,500 

24.6 

26.6 

-2.0 

0.416 

1 to  2 

16.8 

17.6 

-0.9 

0.700 

3 to  4 

11.7 

12.3 

-0.6 

0.745 

5 to  8 

20.6 

18.7 

1.9 

0.420 

9 to  12 

21.1 

22.6 

-1.5 

0.511 

Comnarison  of  uuarter  with  highest  earninss  in  Year  1 
and  auarter  with  hishest  earninss  in  Year  3 

Not  employed  in  either  Year  1 or  Year  3 

5.2 

3.0 

2.2  * 

0.058 

Earnings  decreased  (%) 

No  longer  employed 

15.2 

17.0 

-1.8 

0.414 

Earnings  decreased  by  less  than  $250 

4.1 

2.9 

1.3 

0.250 

Earnings  decreased  by  $250  or  more 

23.2 

24.1 

-0.8 

0.739 

Earnings  increased  (%) 

Became  employed 

4.4 

3.9 

0.5 

0.649 

Earnings  increased  by  less  than  $250 

2.6 

4.8 

-2.1  * 

0.056 

Earnings  increased  by  $250  or  more 

45.2 

44.5 

0.8 

0.793 

Average  earnings  per  quarter  employed  ($) 

3,498 

3,600 

-102 

Sample  size  (total  = 1,164) 

590 

574 

(continued) 
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Appendix  Table  C.ll  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random  assignment, 
even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given  to  those  who  were  not  employed 
in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.12 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Medford 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

93.8 

90.1 

3.7 

0.222 

Currently  employed 

74.1 

70.4 

3.6 

0.466 

No  longer  employed 

19.7 

19.7 

0.0 

0.993 

Current  working  status  (%) 

Full  time 

57.3 

55.2 

2.1 

0.703 

Part  time 

16.7 

15.2 

1.5 

0.720 

Currently  employed  at  a "good  job"a  (%) 

38.8 

30.4 

8.4 

0.100 

Hours 

Average  hours  per  week 

26.7 

24.1 

2.6 

0.206 

Total  hours  per  week  (%) 

Less  than  30 

16.7 

15.2 

1.5 

0.720 

30-34 

9.1 

12.8 

-3.7 

0.301 

35-44 

36.7 

35.2 

1.5 

0.775 

45  or  more 

11.5 

7.2 

4.3 

0.184 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5 .00 

3.6 

2.8 

0.7 

0.715 

$5.00  - $6.99 

4.1 

2.9 

1.1 

0.584 

$7.00  - $8.99 

30.2 

38.6 

-8.4 

0.115 

$9.00  or  more 

36.3 

26.1 

10.2  ** 

0.033 

Average  weekly  earnings  ($) 

254 

212 

4 1 ** 

0.041 

Total  earnings  per  week  (%) 

Less  than  $200 

13.9 

14.5 

-0.7 

0.861 

$201-$300 

16.8 

21.3 

-4.6 

0.292 

$301-$500 

35.0 

31.7 

3.3 

0.517 

$500  or  more 

8.4 

2.8 

5.6  ** 

0.020 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

28.8 

25.2 

3.6 

0.449 

Paid  vacation 

41.2 

37.2 

4.0 

0.433 

Paid  holidays  other  than  Christmas  and  New  Year 

39.3 

36.2 

3.1 

0.546 

Dental  benefits 

37.9 

30.1 

7.8 

0.106 

A retirement  plan 

30.8 

30.1 

0.7 

0.891 

A health  plan  or  medical  insurance 

39.4 

34.4 

5.0 

0.313 

(continued) 
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Appendix  Table  C.12  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Schedule1"  (%) 

Regular 

52.1 

39.9 

12.3  ** 

0.025 

Split  shift 

1.7 

3.4 

-1.7 

0.346 

Irregular 

6.3 

8.7 

-2.5 

0.410 

Evening  shift 

5.2 

8.6 

-3.4 

0.245 

Night  shift 

3.0 

2.8 

0.1 

0.953 

Rotating  shift 

3.9 

4.8 

-0.9 

0.698 

Other  schedule 

1.9 

1.6 

0.2 

0.877 

Odd  job 

Job  skills 

0.0 

0.6 

-0.6 

0.355 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

60.6 

61.7 

-1.1 

0.837 

Work  with  computers 

41.4 

35.9 

5.6 

0.294 

Arithmetic  skills 

52.1 

47.2 

5.0 

0.369 

Customer  contact 

67.6 

65.3 

2.3 

0.664 

Sample  size  (total  = 345) 167 178 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.13 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

Riverside  PASS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

52.2 

55.2 

-3.0 

0.664 

Average  number  of  contacts  with  staff 

9.5 

6.9 

2.7 

0.410 

Talked  with  staff  manager  in  past  4 weeks  (%) 

22.4 

23.2 

-0.8 

0.899 

Staff  ever  talked  with  respondent's  employer  (%) 

5.9 

8.0 

-2.1 

0.569 

Services 

Received  help  with  support  services  (%) 

49.0 

50.2 

-1.2 

0.858 

Received  help  with  basic  needs  (%) 

35.1 

31.6 

3.5 

0.601 

Received  help  with  public  benefits  (%) 

52.6 

58.5 

-5.9 

0.410 

Received  help  with  job  preparation  (%) 

38.3 

28.0 

10.3 

0.119 

Received  help  with  retention/advancement  (%) 

24.4 

20.0 

4.4 

0.454 

Finding  a better  job  while  working 

7.6 

5.7 

1.8 

0.606 

Other  activities  while  working 

5.0 

4.8 

0.2 

0.939 

Career  assessment 

17.9 

6.3 

11.6 

**  0.013 

Dealing  with  problems  on  the  job 

6.7 

6.7 

0.0 

0.998 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

6.3 

7.2 

-0.9 

0.805 

ParticiDation 

Participated  in  any  activity  (%) 

76.9 

70.0 

6.9 

0.251 

Participated  in  any  employment-related  activity3  (%) 

62.1 

55.3 

6.7 

0.319 

Participated  in  a job  search  activity  (%) 

62.2 

53.3 

8.9 

0.190 

Group  job  search/job  club 

35.7 

31.9 

3.7 

0.565 

Individual  job  search 

48.2 

45.3 

3.0 

0.674 

Participated  in  an  education/training  activity  (%) 

42.1 

39.9 

2.2 

0.749 

ABE/GED  and  ESL 

15.9 

12.5 

3.4 

0.473 

College  courses 

20.8 

18.3 

2.4 

0.651 

Vocational  training 

10.7 

13.6 

-2.9 

0.521 

Participated  in  unpaid  work/subsidized  employment  (%) 

2.7 

6.5 

-3.7 

0.209 

Participated  in  an  education  activity  while  working  (%) 

31.0 

27.7 

3.3 

0.599 

Participated  in  an  employment  activity  while  working  (%) 

17.8 

13.1 

4.8 

0.351 

Average  number  of  weeks  participating  in 

Job  search  activities 

5.1 

4.4 

0.7 

0.574 

Education/training  activities 

9.6 

9.4 

0.1 

0.955 

Unpaid  work/subsidized  employment 

0.6 

1.2 

-0.6 

0.410 

Sample  size  (total  = 224) 

120 

104 

(continued) 
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Appendix  Table  C.13  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.14 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

Riverside  PASS 

ERA  Control  Difference  Standard 

Outcome  Group  Group  (Impact)  P-Value  Error 


Follow-Up  Year  1 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  2 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  3 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  4 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 


80.0 

77.2 

2.8  * 

0.057 

1.5 

64.5 

61.9 

2.6  * 

0.061 

1.4 

47.9 

45.2 

2.8 

0.123 

1.8 

40.9 

37.7 

3.2  * 

0.060 

1.7 

9,121 

8,269 

852  *** 

0.003 

287 

41.4 

44.2 

-2.7 

0.143 

1.9 

47.2 

47.9 

-0.7 

0.701 

1.9 

11,671 

10,808 

863  *** 

0.002 

281 

74.2 

69.4 

/J.  g *** 

0.004 

1.7 

59.7 

55.5 

4 2 *** 

0.007 

1.6 

44.1 

40.9 

3.2  * 

0.087 

1.9 

40.1 

36.1 

4.0  ** 

0.026 

1.8 

9,254 

8,385 

869  ** 

0.015 

356 

34.2 

35.2 

-1.0 

0.605 

1.8 

41.6 

43.2 

-1.7 

0.372 

1.9 

11,562 

10,822 

741  ** 

0.033 

348 

71.3 

67.1 

4.3  ** 

0.014 

1.7 

57.7 

54.3 

3.4  ** 

0.035 

1.6 

43.9 

40.6 

3.4  * 

0.073 

1.9 

40.2 

36.0 

4.2  ** 

0.021 

1.8 

9,857 

9,076 

782  * 

0.054 

405 

26.2 

28.4 

-2.3 

0.190 

1.7 

33.8 

35.4 

-1.6 

0.368 

1.8 

11,922 

11,318 

603 

0.131 

399 

68.5 

65.2 

3.3  * 

0.065 

1.8 

56.8 

53.4 

3.4  ** 

0.041 

1.7 

43.9 

40.2 

^ ~j  ** 

0.048 

1.9 

41.7 

37.9 

3.8  ** 

0.040 

1.8 

10,610 

9,643 

967  ** 

0.032 

451 

22.4 

22.8 

-0.3 

0.836 

1.6 

30.1 

32.2 

-2.1 

0.229 

1.8 

12,448 

11,493 

955  ** 

0.032 

445 

Sample  size  (total  = 2,770) 1,627  1,143 

(continued) 
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Appendix  Table  C.14  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.15 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held 

in  the  quarter  after  random  assignment2 

4.0 

3.7 

0.3 

0.117 

Average  quarterly  employment  (%) 

59.7 

56.3 

3 4 *** 

0.006 

Percentage  of  quarters  employed  (%) 

Never  employed 

9.5 

11.2 

-1.7 

0.136 

1-25 

15.9 

18.1 

-2.2 

0.137 

26-50 

15.8 

15.2 

0.6 

0.702 

51-75 

17.8 

18.6 

-0.8 

0.602 

76-100 

41.1 

36.9 

4. 1 ** 

0.022 

Employed  entire  follow-up  period  (%) 

23.7 

20.3 

3.4  ** 

0.029 

Had  employment  spell  of  at  least  4 quarters  (%) 

71.9 

69.4 

2.4 

0.146 

Average  number  of  employers  during  follow-up  period 

3.6 

3.5 

0.1 

0.616 

Number  of  employers  (%) 

Never  employed 

9.5 

11.2 

-1.7 

0.136 

1 to  2 

35.4 

36.6 

-1.2 

0.532 

3 to  4 

26.0 

23.5 

2.5 

0.156 

5 to  8 

22.2 

21.3 

0.9 

0.564 

More  than  8 

6.9 

7.5 

-0.5 

0.611 

Average  number  of  quarters  in  first  employment  spell 

7.1 

6.5 

0.6  *** 

0.004 

Number  of  quarters  until  first  employment  spell 

2.4 

2.8 

o 

4^ 

* 

* 

0.017 

Average  number  of  employment  spells 

1.5 

1.5 

0.0 

0.954 

Average  length,  longest  employment  spell  (quarters) 

8.1 

7.6 

0.6  *** 

0.006 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

9.5 

11.2 

-1.7 

0.136 

1 to  2 

12.4 

13.5 

-1.1 

0.422 

3 to  4 

12.0 

13.2 

-1.2 

0.363 

5 to  8 

21.5 

23.1 

-1.6 

0.345 

9 to  12 

14.8 

12.2 

2.6  * 

0.062 

13  to  16 

29.8 

26.9 

3.0  * 

0.080 

(continued) 


385 


Appendix  Table  C.15  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Average  number  of  unemployment  spells 

1.3 

1.4 

0.0 

0.205 

Length  of  longest  unemployment  spell,  in  quarters 

5.4 

6.0 

-0.6  *** 

0.002 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

23.7 

20.3 

3.4  ** 

0.029 

1 to  2 

17.6 

18.6 

-1.0 

0.519 

3 to  4 

13.9 

12.7 

1.2 

0.382 

5 to  8 

17.7 

17.1 

0.6 

0.690 

9 to  12 

11.5 

12.1 

-0.7 

0.591 

13  to  16 

15.6 

19.1 

-3.5  ** 

0.012 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

9,711 

8,843 

868  *** 

0.006 

Average  annual  earnings  (%) 

$0 

9.5 

11.2 

-1.7 

0.136 

$l-$  1,999 

17.7 

17.1 

0.6 

0.694 

$2,000- $4,999 

14.6 

16.4 

-1.8 

0.213 

$5,000- $9, 999 

17.0 

20.3 

-3.4  ** 

0.030 

$10,000-$  14,999 

14.0 

12.7 

1.4 

0.308 

$15,000-$19,999 

12.6 

9.0 

3.6  *** 

0.004 

$20,000  or  higher 

14.6 

13.3 

1.3 

0.328 

Average  annual  earnings  of  $10,000  or  more  (%) 

41.3 

35.0 

6.3  *** 

0.000 

Quarters  with  earnings  of  $3,500  or  more  (%) 

33.5 

29.5 

q 1 *** 

0.001 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

9.5 

11.2 

-1.7 

0.136 

No  quarters  with  earnings  above  $3,500 

24.2 

24.2 

0.0 

0.979 

1 to  2 

12.9 

14.9 

-2.0 

0.148 

3 to  4 

9.6 

9.3 

0.3 

0.827 

5 to  8 

13.8 

16.1 

-2.3 

0.110 

9 to  12 

11.6 

10.1 

1.5 

0.232 

13  to  16 

18.4 

14.2 

4 3 *** 

0.002 

Comnarison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

12.1 

14.6 

-2.5  ** 

0.048 

Earnings  decreased  (%) 

No  longer  employed 

19.4 

20.2 

-0.8 

0.610 

Earnings  decreased  by  less  than  $250 

2.0 

2.4 

-0.4 

0.459 

Earnings  decreased  by  $250  or  more 

17.2 

15.7 

1.5 

0.317 

Earnings  increased  (%) 

Became  employed 

7.9 

8.2 

-0.3 

0.752 

Earnings  increased  by  less  than  $250 

3.0 

1.9 

1.1  * 

0.083 

Earnings  increased  by  $250  or  more 

38.1 

36.7 

1.4 

0.457 

Average  earnings  per  quarter  employed  ($) 

4,069 

3,931 

137 

Sample  size  (total  = 2,770) 

1,627 

1,143 

(continued) 
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Appendix  Table  C.15  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random  assignment, 
even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given  to  those  who  were  not  employed 
in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.16 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

EmDlovment  status 

Ever  employed  since  random  assignment  (%) 

84.7 

88.8 

-4.1 

0.359 

Currently  employed 

66.3 

76.3 

-10.0 

0.102 

No  longer  employed 

18.4 

12.5 

5.9 

0.251 

Current  working  status  (%) 

Full  time 

56.9 

63.3 

-6.4 

0.334 

Part  time 

9.5 

13.1 

-3.6 

0.413 

Currently  employed  at  a "good  job"a  (%) 

32.5 

41.4 

-9.0 

0.160 

Hours 

Average  hours  per  week 

24.8 

29.2 

-4.4  * 

0.094 

Total  hours  per  week  (%) 

Less  than  30 

9.5 

13.1 

-3.6 

0.413 

30-34 

9.4 

5.5 

3.9 

0.304 

35-44 

38.9 

44.6 

-5.7 

0.395 

45  or  more 

8.6 

13.2 

-4.5 

0.306 

Earninss 

Average  hourly  wage  (%) 

Less  than  $5.00 

3.4 

0.9 

2.5 

0.226 

$5.00  - $6.99 

5.4 

11.1 

-5.7 

0.135 

$7.00  - $8.99 

29.6 

27.4 

2.2 

0.736 

$9.00  or  more 

28.0 

37.0 

-9.0 

0.156 

Average  weekly  earnings  ($) 

222 

272 

-50  * 

0.063 

Total  earnings  per  week  (%) 

Less  than  $200 

10.8 

11.5 

-0.7 

0.875 

$201 -$300 

15.2 

16.1 

-0.8 

0.872 

$301-$500 

32.3 

39.7 

-7.4 

0.249 

$500  or  more 

8.0 

9.0 

-1.0 

0.792 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

26.3 

31.2 

-4.8 

0.431 

Paid  vacation 

36.9 

43.0 

-6.0 

0.354 

Paid  holidays  other  than  Christmas  and  New  Year 

32.9 

42.8 

-9.9 

0.125 

Dental  benefits 

28.3 

33.7 

-5.5 

0.387 

A retirement  plan 

22.7 

29.6 

-6.9 

0.248 

A health  plan  or  medical  insurance 

33.2 

36.7 

-3.5 

0.592 

(continued) 
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Appendix  Table  C.16  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Schedule1"  (%) 

Regular 

36.2 

44.8 

-8.6 

0.219 

Split  shift 

0.9 

-0.1 

1.0 

0.283 

Irregular 

7.7 

3.6 

4.2 

0.189 

Evening  shift 

5.4 

9.2 

-3.8 

0.296 

Night  shift 

6.7 

7.6 

-0.9 

0.813 

Rotating  shift 

7.1 

8.2 

-1.1 

0.779 

Other  schedule 

1.5 

3.1 

-1.6 

0.449 

Odd  job 

Job  skills 

0.8 

0.1 

0.7 

0.494 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

45.2 

60.3 

-15.1  ** 

0.025 

Work  with  computers 

33.7 

37.1 

-3.4 

0.587 

Arithmetic  skills 

42.5 

47.1 

-4.5 

0.497 

Customer  contact 

54.5 

57.3 

-2.9 

0.668 

Sample  size  (total  = 224) 120 104 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“This  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.17 

Impacts  on  Current  Job  Characteristics  and  Advancement  at  the 
42-Month  Survey  for  the  Early  Cohort: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

97.7 

96.8 

0.9 

0.643 

Currently  employed 

61.1 

68.0 

-6.8 

0.184 

t 

No  longer  employed 

36.5 

28.8 

7.7 

0.129 

t 

Hourlv  waee 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.7 

1.3 

0.5 

0.765 

$5.00 -$6.99 

3.3 

8.3 

-5.1  * 

0.079 

$7.00  - $8.99 

15.1 

16.4 

-1.3 

0.765 

$9.00  or  more 

41.6 

42.6 

-1.0 

0.865 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

10.7 

8.1 

2.7 

0.427 

30-34 

6.5 

10.5 

-4.0 

0.208 

35-44 

35.6 

36.7 

-1.2 

0.819 

45  or  more 

8.1 

12.7 

-4.6 

0.175 

tt 

Schedule 

Typical  schedule  (%) 

Regular 

42.3 

46.9 

-4.7 

0.394 

t 

Evening  shift 

3.0 

6.1 

-3.2 

0.182 

Other  schedule 

13.9 

12.5 

1.5 

0.705 

Workweek  included  at  least  1 weekend  day  (%) 

23.6 

32.5 

-8.9  * 

0.075 

Benefits 

Employer  offers  (%) 

Sick  days  with  frill  pay 

31.0 

34.3 

-3.2 

0.537 

Paid  vacation 

37.7 

40.7 

-3.0 

0.582 

Paid  holidays  other  than  Christmas  and  New  Year 

35.8 

38.0 

-2.3 

0.674 

A health  plan  or  medical  insurance 

35.1 

43.0 

-8.0 

0.135 

None  of  the  above 

18.6 

16.3 

2.3 

0.606 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

50.5 

55.3 

-4.8 

0.372 

Receive  respect  from  coworkers 

54.1 

59.4 

-5.3 

0.336 

t 

Receive  proper  equipment  needed  to  do  job 

56.3 

62.5 

-6.2 

0.251 

Are  allowed  to  contribute  ideas 

53.2 

60.8 

-7.6 

0.164 

Can  count  on  keeping  job 

23.7 

18.6 

5.1 

0.270 

Think  job  requires  a lot  of  responsibility 

56.4 

62.9 

-6.5 

0.213 

Think  job  is  physically  demanding 

29.0 

35.4 

-6.5 

0.227 

Risk  health  or  safety 

20.0 

21.4 

-1.4 

0.762 

(continued) 
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Appendix  Table  C.17  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Are  members  of  labor  union  (%) 

9.8 

12.5 

-2.6 

0.456 

Have  possibility  of  a promotion  (%) 

38.9 

44.9 

-6.1 

0.262 

Duties/reauirements.  at  least  once  Der  month  1%  reporting) 

Reading  and  writing  skills 

42.5 

55.9 

-13.4  ** 

0.014 

ttt 

Work  with  computers 

30.3 

38.8 

-8.5 

0.100 

tt 

Arithmetic  skills 

35.2 

38.2 

-3.1 

0.555 

t 

Customer  contact 

50.1 

61.1 

-10.9  ** 

0.039 

t 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

36.5 

28.8 

7.7 

0.129 

t 

0-15 

31.0 

33.6 

-2.6 

0.611 

16-30 

18.5 

19.3 

-0.8 

0.857 

31-45 

7.7 

7.6 

0.0 

0.991 

46  or  more 

3.8 

7.5 

-3.7 

0.154 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

49.2 

49.1 

0.1 

0.983 

Ever  received  a promotion  (%) 

26.6 

31.2 

-4.5 

0.372 

Found  a different  job  while  working  (%) 

39.2 

29.8 

9.5  * 

0.084 

Left  a job  to  go  into  a higher-paying  one  (%) 

31.6 

28.2 

3.5 

0.509 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

40.3 

36.1 

4.2 

0.448 

Earnings 

37.7 

35.0 

2.8 

0.615 

Benefits 

26.7 

21.8 

4.8 

0.325 

Number  of  hours 

36.2 

34.8 

1.4 

0.793 

Start  and  end  of  workday 

38.1 

39.0 

-0.9 

0.870 

Commuting  time 

28.7 

24.8 

3.8 

0.449 

Job  security 

32.2 

34.2 

-2.0 

0.712 

Opportunity  to  advance 

32.7 

31.7 

1.0 

0.849 

Sample  size  (total  = 369) 

207 

162 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across  different 
subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  tt  = 5 percent;  f = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.18 

Impacts  on  Current  Job  Characteristics  and  Advancement  at  the 
42-Month  Survey  for  the  Late  Cohort: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Emolovment  status 

Ever  employed  since  random  assignment  (%) 

95.0 

95.9 

-0.9 

0.658 

Currently  employed 

62.8 

58.5 

4.4 

0.323  f 

No  longer  employed 

32.2 

37.4 

-5.2 

0.224  f 

Hourlv  waee 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.5 

1.0 

0.5 

0.616 

$5.00 -$6.99 

3.8 

9.1 

-5.3  ** 

0.020 

$7.00  - $8.99 

13.0 

12.1 

0.9 

0.789 

$9.00  or  more 

43.6 

34.7 

8.9  * 

0.050 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

9.3 

11.0 

-1.7 

0.538 

30-34 

6.3 

7.3 

-0.9 

0.688 

35-44 

34.0 

32.9 

1.0 

0.814 

45  or  more 

13.1 

7.3 

5.8  ** 

0.043  ff 

Schedule 

Typical  schedule  (%) 

Regular 

42.9 

35.1 

7.8  * 

0.085  f 

Evening  shift 

6.1 

6.7 

-0.6 

0.782 

Other  schedule 

12.4 

15.6 

-3.2 

0.311 

Workweek  included  at  least  1 weekend  day  (%) 

30.6 

30.3 

0.3 

0.948 

Benefits 

Employer  offers  (%) 

Sick  days  with  frill  pay 

33.9 

28.5 

5.4 

0.207 

Paid  vacation 

40.4 

35.1 

5.2 

0.243 

Paid  holidays  other  than  Christmas  and  New  Year 

37.3 

32.6 

4.7 

0.286 

A health  plan  or  medical  insurance 

39.6 

39.3 

0.2 

0.960 

None  of  the  above 

15.1 

12.9 

2.2 

0.507 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

51.0 

48.7 

2.2 

0.627 

Receive  respect  from  coworkers 

55.1 

48.3 

6.8 

0.141  f 

Receive  proper  equipment  needed  to  do  job 

57.9 

55.3 

2.6 

0.559 

Are  allowed  to  contribute  ideas 

52.3 

50.3 

2.0 

0.660 

Can  count  on  keeping  job 

20.9 

19.4 

1.5 

0.690 

Think  job  requires  a lot  of  responsibility 

59.8 

55.9 

3.9 

0.386 

Think  job  is  physically  demanding 

31.7 

32.0 

-0.3 

0.949 

Risk  health  or  safety 

24.7 

21.4 

3.3 

0.396 

(continued) 
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Appendix  Table  C.18  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Are  members  of  labor  union  (%) 

11.4 

12.6 

-1.2 

0.690 

Have  possibility  of  a promotion  (%) 

38.4 

37.5 

0.9 

0.839 

Duties/reauirements.  at  least  once  Der  month  1%  reporting) 

Reading  and  writing  skills 

47.3 

42.4 

4.9 

0.275  fff 

Work  with  computers 

33.7 

25.6 

8.2  ** 

0.047  ff 

Arithmetic  skills 

36.5 

28.0 

8.6  ** 

0.045  f 

Customer  contact 

55.3 

52.8 

2.5 

0.582  f 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

32.2 

37.4 

-5.2 

0.224  f 

0-15 

33.1 

33.2 

-0.2 

0.967 

16-30 

22.5 

17.0 

5.5 

0.141 

31-45 

3.3 

5.5 

-2.1 

0.253 

46  or  more 

3.6 

2.4 

1.3 

0.441 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

46.8 

44.7 

2.1 

0.640 

Ever  received  a promotion  (%) 

29.3 

25.9 

3.5 

0.398 

Found  a different  job  while  working  (%) 

35.1 

34.0 

1.1 

0.798 

Left  a job  to  go  into  a higher-paying  one  (%) 

28.2 

33.8 

-5.6 

0.189 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

37.1 

34.7 

2.4 

0.591 

Earnings 

38.2 

36.0 

2.2 

0.621 

Benefits 

30.6 

30.8 

-0.3 

0.952 

Number  of  hours 

33.7 

28.5 

5.2 

0.218 

Start  and  end  of  workday 

38.9 

32.1 

6.8 

0.126 

Commuting  time 

27.4 

26.8 

0.6 

0.887 

Job  security 

34.9 

29.2 

5.7 

0.192 

Opportunity  to  advance 

35.6 

28.1 

7.6  * 

0.080 

Sample  size  (total  = 523) 

297 

226 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42-Month  Survey  are  presented  in 
Appendix  D. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across  different 
subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  tt  = 5 percent;  f = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.19 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4),  by  Type  of  Service  Provider: 

Riverside  PASS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Communitv-based  organization 

Employment  retention  measures 

Ever  employed  (%) 

90.6 

89.0 

1.6 

0.237 

Average  quarterly  employment  (%) 

59.3 

54.5 

4 g *** 

0.001 

t 

Had  employment  spell  of  at  least  4 quarters  (%) 

71.9 

67.4 

4.5  ** 

0.024 

tt 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

9,627 

8,445 

1,182  *** 

0.001 

t 

Average  annual  earnings  of  $ 1 0,000  or  more  (%) 

41.0 

32.6 

g ^ *** 

0.000 

tt 

Quarters  with  earnings  of  $3,500  or  more  (%) 

33.5 

28.0 

5 5 *** 

0.000 

tt 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

1,296 

1,307 

-ii 

0.894 

Average  annual  food  stamps  received  ($) 

961 

989 

-28 

0.615 

Average  annual  income  ($) 

11,884 

10,741 

1,143  *** 

0.001 

tt 

Sample  size  (total  = 2,068) 

1,246 

822 

Communitv  college 

Employment  retention  measures 

Ever  employed  (%) 

91.4 

88.9 

2.5 

0.276 

Average  quarterly  employment  (%) 

60.9 

61.3 

-0.4 

0.877 

t 

Had  employment  spell  of  at  least  4 quarters  (%) 

72.6 

76.2 

-3.6 

0.294 

tt 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

9,894 

10,260 

-366 

0.612 

t 

Average  annual  earnings  of  $ 1 0,000  or  more  (%) 

40.6 

42.8 

-2.2 

0.552 

tt 

Quarters  with  earnings  of  $3,500  or  more  (%) 

33.7 

34.4 

-0.7 

0.797 

tt 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

1,095 

1,198 

-103 

0.464 

Average  annual  food  stamps  received  ($) 

842 

898 

-56 

0.542 

Average  annual  income  ($) 

11,832 

12,356 

-525 

0.450 

tt 

Sample  size  (total  = 590) 

295 

295 

SOURCE:  MDRC  calculations  from  state  unemployment  insurance  (UI)  wage  records. 


NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
different  subgroups.  Statistical  significance  levels  are  indicated  as  follows:  f ff  = 1 percent;  ft  = 5 percent;  f = 
1 0 percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.20 

Impacts  on  Current  Job  Characteristics  and  Advancement  at  the  42-Month  Survey  for 
Study  Participants  Eligible  to  Be  Served  by  a Community-Based  Organization: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

EmDlovment  status 

Ever  employed  since  random  assignment  (%) 

96.0 

95.8 

0.2 

0.907 

Currently  employed 

60.1 

62.9 

-2.8 

0.464 

No  longer  employed 

35.9 

32.9 

3.0 

0.425 

t 

Hourlv  wage 

Average  hourly  wage  (%) 

Less  than  $5.00 

1.5 

1.1 

0.4 

0.665 

$5.00  - $6.99 

4.0 

10.3 

-6.3  *** 

0.003 

$7.00  - $8.99 

14.4 

13.9 

0.5 

0.866 

$9.00  or  more 

40.6 

36.5 

4.1 

0.308 

Hours 

Average  hours  per  week  (%) 

Less  than  30 

9.5 

11.3 

-1.8 

0.468 

30-34 

5.9 

8.7 

-2.8 

0.184 

35-44 

35.1 

33.0 

2.1 

0.582 

45  or  more 

9.5 

9.9 

-0.4 

0.860 

t 

Schedule 

Typical  schedule  (%) 

Regular 

42.1 

39.8 

2.4 

0.554 

Evening  shift 

4.4 

6.9 

-2.5 

0.186 

Other  schedule 

11.5 

15.2 

-3.6 

0.185 

Workweek  included  at  least  1 weekend  day  (%) 

26.7 

32.1 

-5.3 

0.152 

Benefits 

Employer  offers  (%) 

Sick  days  with  full  pay 

32.3 

29.8 

2.5 

0.516 

Paid  vacation 

38.3 

36.8 

1.5 

0.705 

Paid  holidays  other  than  Christmas  and  New  Year 

36.1 

33.1 

2.9 

0.449 

A health  plan  or  medical  insurance 

37.4 

41.4 

4.0 

0.307 

None  of  the  above 

16.2 

15.1 

1.1 

0.711 

Work  environment 

Percentage  who  agreed  that  they: 

Receive  respect  from  superiors 

49.7 

51.7 

-2.0 

0.619 

Receive  respect  from  coworkers 

53.1 

52.2 

1.0 

0.812 

Receive  proper  equipment  needed  to  do  job 

54.9 

58.3 

-3.4 

0.389 

Are  allowed  to  contribute  ideas 

51.2 

54.6 

-3.4 

0.395 

Can  count  on  keeping  job 

21.1 

19.9 

1.2 

0.704 

Think  job  requires  a lot  of  responsibility 

56.6 

58.6 

-2.0 

0.615 

Think  job  is  physically  demanding 

29.1 

35.2 

-6.1 

0.110 

t 

Risk  health  or  safety 

22.7 

20.1 

2.6 

0.434 

(continued) 


395 


Appendix  Table  C.20  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Are  members  of  labor  union  (%) 

11.7 

13.1 

-1.4 

0.591 

Elave  possibility  of  a promotion  (%) 

37.9 

39.5 

-1.6 

0.689 

Duties/reauirements.  at  least  once  Der  month  t%  renortinsl 

Reading  and  writing  skills 

44.3 

48.8 

-4.5 

0.256 

Work  with  computers 

30.6 

32.4 

-1.8 

0.614 

Arithmetic  skills 

34.5 

33.6 

0.9 

0.817 

Customer  contact 

51.9 

55.9 

-4.0 

0.311 

Transnortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

35.9 

32.9 

3.0 

0.425 

t 

0-15 

33.9 

32.3 

1.5 

0.688 

16-30 

18.2 

18.9 

-0.7 

0.820 

31-45 

3.8 

7.0 

-3.1  * 

0.087 

46  or  more 

4.0 

4.6 

-0.6 

0.728 

Advancement  outcomes  since  random  assignment 

Ever  received  a raise  (%) 

49.8 

46.1 

3.7 

0.355 

t 

Ever  received  a promotion  (%) 

29.3 

29.6 

-0.3 

0.927 

Found  a different  job  while  working  (%) 

36.9 

36.7 

0.3 

0.943 

Left  a job  to  go  into  a higher-paying  one  (%) 

30.2 

31.0 

-0.8 

0.832 

Compared  with  previous  jobs  since  random  assignment, 
percentage  who  reported  current  job  improvements  in  (%) 

Work  enjoyment 

39.3 

34.3 

5.0 

0.212 

Earnings 

39.3 

35.4 

3.9 

0.331 

Benefits 

31.5 

26.5 

5.0 

0.179 

tt 

Number  of  hours 

36.2 

30.0 

6.2 

0.108 

Start  and  end  of  workday 

41.1 

35.1 

6.0 

0.131 

Commute  time 

29.0 

28.0 

1.1 

0.768 

Job  security 

34.7 

32.0 

2.7 

0.484 

Opportunity  to  advance 

35.7 

31.1 

4.6 

0.239 

Sample  size  (total  = 690) 

404 

286 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Industry  type,  firm  size,  occupation,  and  other  measures  from  the  ERA  42 -Month  Survey  are  presented  in 
Appendix  D. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across  different 
subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  (t  = 5 percent;  | = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.21 

Impacts  on  Contacts,  Services,  and  Participation  at  the  12-Month  Survey: 

South  Carolina 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Contacts 

Ever  had  contact  with  staff/employment  program  (%) 

43.0 

29.1 

13  9 *** 

0.001 

Average  number  of  contacts  with  staff 

4.4 

2.7 

1.7  * 

0.077 

Talked  with  staff  manager  in  past  4 weeks  (%) 

16.0 

10.8 

5.3  * 

0.077 

Staff  ever  talked  with  respondent's  employer  (%) 

5.9 

3.3 

2.6 

0.164 

Services 

Received  help  with  support  services  (%) 

21.9 

13.7 

g 9 *** 

0.010 

Received  help  with  basic  needs  (%) 

29.2 

29.5 

-0.4 

0.922 

Received  help  with  public  benefits  (%) 

55.3 

57.4 

-2.1 

0.625 

Received  help  with  job  preparation  (%) 

20.8 

16.6 

4.2 

0.204 

Received  help  with  retention/advancement  (%) 

17.6 

8.2 

9 4 *** 

0.001 

Finding  a better  job  while  working 

12.8 

3.9 

g 9 *** 

0.000 

Other  activities  while  working 

6.5 

3.3 

3.2  * 

0.080 

Career  assessment 

7.5 

4.1 

3.4  * 

0.094 

Dealing  with  problems  on  the  job 

4.6 

2.3 

2.3 

0.153 

Addressing  a personal  problem  that  makes  it  hard  to  keep  a job 

4.4 

2.5 

2.0 

0.218 

ParticiDation 

Participated  in  any  activity  (%) 

54.3 

53.0 

1.4 

0.747 

Participated  in  any  employment-related  activity3  (%) 

44.5 

44.6 

-0.1 

0.991 

Participated  in  a job  search  activity  (%) 

42.4 

44.2 

-1.9 

0.654 

Group  job  search/job  club 

18.4 

16.1 

2.3 

0.480 

Individual  job  search 

37.3 

38.5 

-1.2 

0.767 

Participated  in  an  education/training  activity  (%) 

26.0 

19.7 

6.3  * 

0.080 

ABE/GED  and  ESL 

10.0 

9.6 

0.4 

0.883 

College  courses 

11.3 

9.3 

2.0 

0.436 

Vocational  training 

8.7 

3.2 

3 3 *** 

0.006 

Participated  in  unpaid  work/subsidized  employment  (%) 

5.7 

2.6 

3.1  * 

0.072 

Participated  in  an  education  activity  while  working  (%) 

14.2 

9.0 

5.3  * 

0.055 

Participated  in  an  employment  activity  while  working  (%) 

12.5 

7.8 

4.7  * 

0.071 

Average  number  of  weeks  participating  in 

Job  search  activities 

4.9 

4.3 

0.6 

0.528 

Education/training  activities 

3.8 

2.4 

1.4  * 

0.083 

Unpaid  work/subsidized  employment 

0.7 

0.4 

0.3 

0.220 

Sample  size  (total  = 552) 

276 

276 

(continued) 
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Appendix  Table  C.21  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  includes  respondents  who  participated  in  individual  or  group  job  searches,  job  clubs,  or  on-the- 
job  training,  or  who  had  an  unpaid  or  subsidized  job. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.22 

Impacts  on  Ul-Covered  Employment,  Public  Assistance,  and  Measured  Income: 

South  Carolina 

ERA  Control  Difference  Standard 

Outcome  Group  Group  Impact  P-Value  Error 

Follow-Up  Year  1 


Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  2 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  3 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANF  (%) 

Ever  received  food  stamps  (%) 

Total  income  ($) 

Follow-Up  Year  4 

Ever  employed  (%) 

Average  quarterly  employment  (%) 

Employed  every  quarter  (%) 

Employed  with  annual  earnings  over  $10,000  (%) 
Annual  earnings  ($) 

Ever  received  TANFa  (%) 

Ever  received  food  stamps'*  (%) 

Total  income2  ($) 


63.2 

63.2 

0.1 

0.972 

1.4 

50.3 

49.7 

0.6 

0.570 

1.1 

37.7 

36.2 

1.5 

0.288 

1.4 

25.7 

25.8 

-0.1 

0.939 

1.3 

5,916 

6,063 

-147 

0.411 

179 

7.5 

7.0 

0.6 

0.577 

1.0 

62.5 

61.5 

1.0 

0.461 

1.3 

7,838 

8,000 

-162 

0.355 

175 

59.0 

58.1 

0.9 

0.553 

1.5 

47.9 

46.5 

1.4 

0.291 

1.3 

36.0 

34.0 

2.0 

0.183 

1.5 

25.4 

26.1 

-0.8 

0.566 

1.4 

5,947 

5,959 

-12 

0.958 

217 

8.4 

8.4 

0.0 

0.974 

1.1 

62.5 

62.6 

0.0 

0.984 

1.5 

8,020 

8,043 

-23 

0.915 

212 

58.4 

56.1 

2.3 

0.146 

1.6 

47.4 

45.6 

1.7 

0.213 

1.4 

35.6 

34.0 

1.6 

0.316 

1.6 

26.6 

25.7 

1.0 

0.500 

1.4 

6,089 

6,095 

-6 

0.981 

248 

7.9 

6.1 

1.7  * 

0.075 

1.0 

61.7 

61.1 

0.6 

0.718 

1.6 

8,212 

8,167 

44 

0.855 

243 

57.6 

57.3 

0.4 

0.826 

1.6 

46.7 

46.0 

0.6 

0.652 

1.4 

34.1 

33.8 

0.3 

0.851 

1.6 

27.1 

26.0 

1.2 

0.413 

1.4 

6,265 

6,367 

-102 

0.706 

271 

6.7 

5.9 

0.8 

0.424 

1.0 

59.2 

58.5 

0.7 

0.677 

1.7 

8,400 

8,532 

-131 

0.632 

275 

Sample  size  (total  = 2,776)  1,382  1,394 

(continued) 
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Appendix  Table  C.22  (continued) 

SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

a156  sample  members  do  not  have  four  years  of  follow-up  on  food  stamps,  TANF,  and  total  income.  Because 
of  this,  the  sum  of  earnings,  TANF,  and  food  stamps  does  not  equal  total  income  in  Year  4. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.23 

Impacts  on  Employment  Retention  and  Advancement 
in  the  Cumulative  Follow-Up  Period  (Years  1-4): 

South  Carolina 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

EmDlovment  measures 

Number  of  quarters  employed  at  jobs  held 

in  the  quarter  after  random  assignment2 

3.5 

3.3 

0.2 

0.341 

Average  quarterly  employment  (%) 

48.1 

47.0 

1.1 

0.303 

Percentage  of  quarters  employed  (%) 

Never  employed 

25.1 

24.9 

0.2 

0.875 

1-25 

15.7 

17.1 

-1.4 

0.295 

26-50 

12.4 

13.0 

-0.6 

0.641 

51-75 

12.5 

12.6 

-0.1 

0.956 

76-100 

34.3 

32.4 

1.9 

0.193 

Employed  entire  follow-up  period  (%) 

18.6 

18.7 

0.0 

0.973 

Had  employment  spell  of  at  least  4 quarters  (%) 

55.6 

55.6 

0.1 

0.974 

Average  number  of  employers  during  follow-up  period 

2.4 

2.4 

0.1 

0.392 

Number  of  employers  (%) 

Never  employed 

25.1 

24.9 

0.2 

0.875 

1 to  2 

36.8 

37.1 

-0.3 

0.852 

3 to  4 

21.0 

21.7 

-0.7 

0.656 

5 to  8 

13.9 

13.2 

0.7 

0.612 

More  than  8 

3.3 

3.2 

0.2 

0.810 

Average  number  of  quarters  in  first  employment  spell 

5.7 

5.6 

0.1 

0.485 

Number  of  quarters  until  first  employment  spell 

4.9 

4.9 

0.0 

0.851 

Average  number  of  employment  spells 

1.3 

1.3 

0.0 

0.726 

Average  length,  longest  employment  spell  (quarters) 

6.5 

6.3 

0.2 

0.281 

Quarters  in  longest  employment  spell  (%) 

Never  employed 

25.1 

24.9 

0.2 

0.875 

1 to  2 

14.1 

14.0 

0.2 

0.907 

3 to  4 

9.1 

10.9 

-1.8 

0.107 

5 to  8 

16.7 

17.6 

-0.9 

0.529 

9 to  12 

11.3 

9.9 

1.4 

0.233 

13  to  16 

23.8 

22.8 

1.0 

0.492 

(continued) 
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Appendix  Table  C.23  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Average  number  of  unemployment  spells 

1.3 

1.3 

0.0 

0.822 

Length  of  longest  unemployment  spell,  in  quarters 

7.3 

7.5 

-0.2 

0.210 

Quarters  in  longest  unemployment  spell  (%) 

Never  unemployed 

18.6 

18.7 

0.0 

0.973 

1 to  2 

16.5 

13.8 

0.039 

3 to  4 

8.8 

9.9 

-1.1 

0.329 

5 to  8 

16.0 

15.4 

0.7 

0.628 

9 to  12 

9.0 

11.2 

-2.2  * 

0.052 

13  to  16 

31.1 

31.1 

0.0 

0.985 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

6,054 

6,121 

-67 

0.733 

Average  annual  earnings  (%) 

$0 

25.1 

24.9 

0.2 

0.875 

$l-$  1,999 

20.6 

21.0 

-0.4 

0.787 

$2,000- $4,999 

12.9 

13.2 

-0.3 

0.811 

$5,000- $9, 999 

15.8 

16.2 

-0.3 

0.810 

$10,000-$  14,999 

12.5 

11.2 

1.3 

0.267 

$15,000-$19,999 

8.3 

7.9 

0.4 

0.675 

$20,000  or  higher 

4.8 

5.7 

-0.9 

0.210 

Average  annual  earnings  of  $10,000  or  more  (%) 

25.6 

24.7 

0.8 

0.531 

Quarters  with  earnings  of  $3,500  or  more  (%) 

24.2 

24.6 

-0.5 

0.686 

Number  of  quarters  earning  above  $3,500  (%) 

Never  employed 

25.1 

24.9 

0.2 

0.875 

No  quarters  with  earnings  above  $3,500 

36.5 

35.5 

1.0 

0.556 

1 to  2 

9.7 

9.7 

0.0 

0.996 

3 to  4 

5.5 

5.8 

-0.3 

0.704 

5 to  8 

7.2 

7.8 

-0.7 

0.506 

9 to  12 

6.5 

7.1 

-0.6 

0.544 

13  to  16 

9.6 

9.2 

0.4 

0.710 

Comnarison  of  auarter  with  highest  earnings  in  Year  1 

and  auarter  with  highest  earnings  in  Year  4 

Not  employed  in  either  Year  1 or  Year  4 

28.5 

28.4 

0.1 

0.936 

Earnings  decreased  (%) 

No  longer  employed 

13.9 

14.4 

-0.5 

0.726 

Earnings  decreased  by  less  than  $250 

3.5 

2.9 

0.6 

0.334 

Earnings  decreased  by  $250  or  more 

16.2 

16.0 

0.2 

0.879 

Earnings  increased  (%) 

Became  employed 

8.3 

8.5 

-0.2 

0.881 

Earnings  increased  by  less  than  $250 

3.8 

3.6 

0.3 

0.725 

Earnings  increased  by  $250  or  more 

25.9 

26.3 

-0.4 

0.780 

Average  earnings  per  quarter  employed  ($) 

3,084 

3,196 

-112 

Sample  size  (total  = 2,776) 

1,382 

1,394 

(continued) 
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Appendix  Table  C.23  (continued) 


SOURCE:  MDRC  calculations  from  state  administrative  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 

“Year  1”  refers  to  Quarters  2 to  5.  Quarter  1 is  the  quarter  in  which  random  assignment  took  place. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  measure  counts  the  number  of  quarters  employed  at  any  jobs  held  in  the  quarter  after  random 
assignment,  even  among  those  who  were  working  at  these  jobs  previously,  eroes  were  given  to  those  who  were 
not  employed  in  this  quarter. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.24 

Impacts  on  Characteristics  of  Current  Job  at  the  12-Month  Survey: 

South  Carolina 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Employment  status 


Ever  employed  since  random  assignment  (%) 

77.3 

75.3 

2.0 

0.578 

Currently  employed 

54.8 

50.6 

4.2 

0.306 

No  longer  employed 

22.5 

24.6 

-2.2 

0.550 

Current  working  status  (%) 

Full  time 

46.7 

44.9 

1.8 

0.653 

Part  time 

8.1 

5.7 

2.3 

0.290 

Currently  employed  at  a "good  job"a  (%) 

18.8 

19.2 

-0.4 

0.893 

Hours 

Average  hours  per  week 

20.1 

19.3 

0.8 

0.633 

Total  hours  per  week  (%) 

Less  than  30 

8.1 

5.7 

2.3 

0.290 

30-34 

9.7 

6.3 

3.4 

0.151 

35-44 

29.4 

30.7 

-1.3 

0.734 

45  or  more 

7.7 

7.9 

-0.3 

0.902 

Earnings 

Average  hourly  wage  (%) 

Less  than  $5.00 

4.3 

5.5 

-1.3 

0.505 

$5.00  - $6.99 

16.7 

18.1 

-1.3 

0.683 

$7.00  - $8.99 

17.9 

14.4 

3.5 

0.267 

$9.00  or  more 

15.9 

12.7 

3.3 

0.240 

Average  weekly  earnings  ($) 

160 

155 

5 

0.730 

Total  earnings  per  week  (%) 

Less  than  $200 

14.2 

13.0 

1.3 

0.673 

$201-$300 

19.2 

19.9 

-0.7 

0.824 

$301-$500 

18.3 

14.0 

4.3 

0.153 

$500  or  more 

3.1 

3.8 

-0.6 

0.665 

Benefits 

Currently  employed  and  receiving  employer-provided 
benefits  at  current  job  (%) 

Sick  days  with  full  pay 

18.8 

21.4 

-2.7 

0.404 

Paid  vacation 

32.2 

31.6 

0.6 

0.865 

Paid  holidays  other  than  Christmas  and  New  Year 

24.6 

26.5 

-1.9 

0.595 

Dental  benefits 

21.4 

25.4 

4.0 

0.226 

A retirement  plan 

20.7 

24.6 

-3.9 

0.230 

A health  plan  or  medical  insurance 

26.0 

30.2 

4.1 

0.229 

(continued) 
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Appendix  Table  C.24  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Schedule"  f%l 

Regular 

31.7 

32.4 

-0.7 

0.865 

Split  shift 

0.7 

0.8 

-0.1 

0.915 

Irregular 

3.4 

2.4 

1.0 

0.493 

Evening  shift 

4.9 

4.8 

0.1 

0.956 

Night  shift 

4.4 

4.3 

0.1 

0.965 

Rotating  shift 

8.5 

4.9 

3.7  * 

0.088 

Other  schedule 

0.4 

0.7 

-0.3 

0.606 

Odd  job 

Job  skills 

0.8 

0.3 

0.4 

0.517 

Percentage  reporting  that  job  requires  each  at  least  monthly 

Reading  and  writing  skills 

37.9 

35.3 

2.7 

0.486 

Work  with  computers 

22.5 

16.6 

5.9  * 

0.062 

Arithmetic  skills 

24.7 

26.4 

-1.6 

0.651 

Customer  contact 

43.7 

41.8 

1.9 

0.639 

Sample  size  (total  = 552) 276 276 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  12-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aThis  definition  of  a good  job  is  adapted  from  Johnson  and  Corcoran  (2003).  A "good  job"  is  a job  in  which 
a respondent  works  35  or  more  hours  per  week  and  either  (1)  pays  $7.00  or  more  per  hour  and  offers  health 
insurance  or  (2)  pays  $8.50  or  more  per  hour. 

bA  split  shift  is  defined  as  one  consisting  of  two  distinct  periods  each  day.  An  irregular  schedule  is  defined 
as  one  that  changes  from  day  to  day.  A rotating  shift  is  one  that  changes  regularly  from  days  to  evenings  to 
nights. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  C.25 

Impacts  on  Employment  Retention,  Advancement,  and  TANF  Receipt 
in  the  Cumulative  Follow-Up  Period  (Years  1-4),  by  Fabor  Force  Attachment  Status: 

South  Carolina 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Not  attached  to  the  labor  market 

Employment  retention  measures 

Ever  employed  (%) 

42.2 

42.4 

-0.2 

0.951 

Average  quarterly  employment  (%) 

14.2 

15.0 

-0.9 

0.588 

Had  employment  spell  of  at  least  4 quarters  (%) 

17.9 

19.9 

-2.0 

0.437 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

1,186 

1,366 

-180 

0.387 

Average  annual  earnings  of  $10,000  or  more  (%) 

3.0 

3.7 

-0.7 

0.529 

Quarters  with  earnings  of  $3,500  or  more 

2.8 

3.6 

-0.8 

0.330 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

74 

65 

9 

0.576 

Average  annual  food  stamps  received  ($) 

1,699 

1,703 

-4 

0.967 

Average  annual  income  ($) 

2,987 

3,153 

-166 

0.478  ff 

Sample  size  ( total  = 953) 

484 

469 

Partiallv  attached  to  the  labor  market 

Employment  retention  measures 

Ever  employed  (%) 

83.9 

85.5 

-1.6 

0.533 

Average  quarterly  employment  (%) 

48.3 

43.7 

4.5  * 

0.052 

Had  employment  spell  of  at  least  4 quarters  (%) 

57.8 

54.2 

3.6 

0.304 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

4,962 

4,331 

631  * 

0.098 

Average  annual  earnings  of  $10,000  or  more  (%) 

17.2 

13.1 

4.1  * 

0.090 

Quarters  with  earnings  of  $3,500  or  more 

12.7 

10.5 

2.2 

0.159 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

100 

90 

10 

0.647 

Average  annual  food  stamps  received  ($) 

2,616 

2,643 

-27 

0.827 

Average  annual  income  ($) 

7,855 

7,031 

825  ** 

0.032  ff 

Sample  size  (total  = 752) 

379 

373 

(continued) 
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Appendix  Table  C.25  (continued) 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Attached  to  the  labor  market 

Employment  retention  measures 

Ever  employed  (%) 

97.5 

97.3 

0.2 

0.851 

Average  quarterly  employment  (%) 

78.2 

77.4 

0.8 

0.650 

Had  employment  spell  of  at  least  4 quarters  (%) 

87.7 

88.2 

-0.5 

0.788 

Earnings  and  advancement  measures 

Average  annual  earnings  ($) 

11,182 

11,568 

-386 

0.322 

Average  annual  earnings  of  $10,000  or  more  (%) 

51.8 

51.3 

0.4 

0.878 

Quarters  with  earnings  of  $3,500  or  more 

35.8 

37.2 

-1.5 

0.409 

Public  assistance  and  income  (Years  1-4) 

Average  annual  TANF  received  ($) 

56 

53 

3 

0.768 

Average  annual  food  stamps  received  ($) 

1,786 

1,746 

41 

0.614 

Average  annual  income  ($) 

12,984 

13,432 

-448 

0.241  ff 

Sample  size  (total  = 1,071) 

519 

552 

SOURCE:  MDRC  calculations  from  state  unemployment  insurance  (UI)  wage  records. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups 
arose  by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  H-statistic  test  was  used  to  test  for  statistically  significant  differences  in  impact  estimates  across 
different  subgroups.  Statistical  significance  levels  are  indicated  as  follows:  ftt  = 1 percent;  ft  = 5 percent; 
f = 10  percent. 

A small  number  of  sample  members  are  missing  TANF  and  food  stamp  data  from  the  last  quarter  of  the 
follow-up  period.  These  people  are  excluded  from  impact  estimates  on  the  following  measures:  Average 
annual  TANF  received  ($);  Average  annual  food  stamps  received  ($);  and  Average  annual  income  ($). 
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Appendix  D 

Exhibits  Showing  Additional  Outcomes  from  the 
42-Month  Survey  and  the 
42-Month  Survey  Response  Analysis 


Appendix  D presents  supplementary  exhibits  from  the  42-month  survey  and  includes  the  42- 
month  survey  response  analysis  for  the  Chicago,  Los  Angeles  Reach  for  Success  (RFS),  and 
Riverside  Post-Assistance  Self-Sufficiency  (PASS)  programs  in  the  Employment  Retention  and 
Advancement  (ERA)  project. 

Table 

D.  1 Impacts  on  Participation  in  Job  Search,  Education,  and  Training  at  the  42 -Month  Survey: 
Chicago 

D.2  Impacts  on  Educational  Attainment  at  the  42 -Month  Survey:  Chicago 

D.3  Impacts  on  Characteristics  of  Current  Job  at  the  42 -Month  Survey:  Chicago 

D.4  Impacts  on  Type  of  Industry,  Type  of  Occupation,  and  Firm  Size  in  Current  Job  at  the 

42 -Month  Survey:  Chicago 

D.5  Impacts  on  Employment  Retention  at  the  42 -Month  Survey:  Chicago 

D.6  Impacts  on  Household  Income  and  Savings  at  the  42 -Month  Survey:  Chicago 

D.7  Impacts  on  Household  Composition,  Housing,  and  Housing  Expenditures  at  the 

42 -Month  Survey:  Chicago 

D.8  Impacts  on  Health  and  Health  Care  Coverage  at  the  42 -Month  Survey:  Chicago 

D.9  Impacts  on  Child  Care  and  Transportation:  Chicago 

D.10  Impacts  on  Child  Care  for  Children  Ages  5 to  8 at  the  42-Month  Survey:  Chicago 
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Table 

D.  1 1 Impacts  on  School  Outcomes  and  Behavioral  Problems  for  Children  Ages  5 to  8 
at  the  42 -Month  Survey:  Chicago 

D.12  Impacts  on  School  Outcomes  for  Children  Ages  13  to  17  at  the  42 -Month  Survey:  Chicago 

D.  1 3 Impacts  on  Behavioral  Problems  and  Police  Involvement  Outcomes  for  Children  Ages 

13  to  17  at  the  42 -Month  Survey 

D.  14  Impacts  on  Participation  in  Job  Search,  Education,  and  Training  at  the  42 -Month 
Survey:  Los  Angeles  RES 

D.  1 5 Impacts  on  Educational  Attainment  at  the  42 -Month  Survey:  Los  Angeles  RES 

D.  16  Impacts  on  Characteristics  of  Current  Job  at  the  42-Month  Survey:  Los  Angeles  RES 

D.  17  Impacts  on  Type  of  Industry,  Type  of  Occupation,  and  Firm  Size  in  Current  Job  at  the 

42 -Month  Survey:  Los  Angeles  RFS 

D.  1 8 Impacts  on  Employment  Retention  at  the  42 -Month  Survey:  Los  Angeles  RES 

D.  19  Impacts  on  Household  Income  and  Savings  at  the  42-Month  Survey:  Los  Angeles  RES 

D.20  Impacts  on  Household  Composition,  Housing,  and  Housing  Expenditures  at  the 
42 -Month  Survey:  Los  Angeles  RFS 

D.21  Impacts  on  Health  and  Health  Care  Coverage  at  the  42 -Month  Survey:  Los  Angles  RFS 

D.22  Impacts  on  Child  Care  and  Transportation  at  the  42 -Month  Survey:  Los  Angeles  RFS 

D.23  Impacts  on  Child  Care  for  Children  Ages  5 to  8 at  the  42 -Month  Survey:  Los  Angeles  RFS 

D.24  Impacts  on  School  Outcomes  and  Behavioral  Problems  for  Children  Ages  5 to  8 
at  the  42 -Month  Survey:  Los  Angeles  RFS 

D.25  Impacts  on  School  Outcomes  for  Children  Ages  13  to  17  at  the  42 -Month  Survey: 

Los  Angeles  RES 

D.26  Impacts  on  Behavioral  Problems  and  Police  Involvement  Outcomes  for  Children  Ages 
13  to  17  at  the  42 -Month  Survey:  Los  Angeles  RFS 

D.27  Impacts  on  Participation  in  Job  Search,  Education,  and  Training  at  the  42-Month 
Survey:  Riverside  PASS 

D.28  Impacts  on  Educational  Attainment  at  the  42 -Month  Survey:  Riverside  PASS 

D.29  Impacts  on  Job  Characteristics  at  the  42 -Month  Survey:  Riverside  PASS 

D.30  Impacts  on  Type  of  Industry,  Type  of  Occupation,  and  Firm  Size  in  Current  Job  at  the 
42 -Month  Survey:  Riverside  PASS 
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Table 

D.3 1 Impacts  on  Employment  Retention  at  the  42 -Month  Survey:  Riverside  PASS 

D.32  Impacts  on  Household  Income  and  Savings  at  the  42 -Month  Survey:  Riverside  PASS 

D.33  Impacts  on  Household  Composition,  Housing,  and  Housing  Expenditures  at  the 
42 -Month  Survey:  Riverside  PASS 

D.34  Impacts  on  Health  and  Health  Care  Coverage  at  the  42 -Month  Survey: 

Riverside  PASS 

D.35  Impacts  on  Child  Care  and  Transportation  at  the  42 -Month  Survey:  Riverside  PASS 

D.36  Impacts  on  Child  Care  for  Children  Ages  5 to  8 at  the  42-Month  Survey: 

Riverside  PASS 

D.37  Impacts  on  School  Outcomes  and  Behavioral  Problems  for  Children  Ages  5 to  8 
at  the  42 -Month  Survey:  Riverside  PASS 

D.38  Impacts  on  School  Outcomes  for  Children  Ages  13  to  17  at  the  42 -Month  Survey: 
Riverside  PASS 

D.39  Impacts  on  Behavioral  Problems  and  Police  Involvement  Outcomes  for  Children  Ages 
13  to  17  at  the  42 -Month  Survey:  Riverside  PASS 

The  42-Month  Survey  Response  Analysis 

Box 

D.  1 Terms  Used  in  the  Survey  Response  Analysis 

D.2  Key  Analysis  Samples:  Chicago 

D.3  Key  Analysis  Samples:  Los  Angles  RFS 

D.4  Key  Analysis  Samples:  Riverside  PASS 

Table 

D.40  Response  Rates:  Chicago 

D.4 1 Estimated  Regression  Coefficients  for  the  Probability  of  Being  a Respondent  to  the 
ERA  42 -Month  Survey:  Chicago 

D.42  Background  Characteristics  of  Respondents  to  the  42 -Month  Survey:  Chicago 

D.43  Comparison  of  Impacts  for  the  Report,  Fielded,  Eligible,  and  Respondent  Samples  at  the 
42 -Month  Survey:  Chicago 

D.44  Response  Rates:  Los  Angeles  RFS 
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Table 

D.45  Estimated  Regression  Coefficients  for  the  Probability  of  Being  a Respondent  to  the 

ERA  42 -Month  Survey:  Los  Angeles  RES 

D.46  Background  Characteristics  of  Respondents  to  the  42-Month  Survey:  Los  Angeles  RFS 

D.47  Comparison  of  Impacts  for  the  Report,  Fielded,  Eligible,  and  Respondent  Samples  at  the 
42 -Month  Survey:  Los  Angeles  RFS 

D.48  Response  Rates:  Riverside  PASS 

D.49  Estimated  Regression  Coefficients  for  the  Probability  of  Being  a Respondent  to  the 

ERA  42 -Month  Survey:  Riverside  PASS 

D.50  Background  Characteristics  of  Respondents  to  the  42-Month  Survey:  Riverside  PASS 

D.5 1 Comparison  of  Impacts  for  the  Report,  Fielded,  Eligible,  and  Respondent  Samples  at  the 
42 -Month  Survey:  Riverside  PASS 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.l 

Impacts  on  Participation  in  Job  Search,  Education,  and  Training 
at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Since  random  assignment  (%) 

Ever  participated  in  any  activity3 

57.3 

55.4 

1.9 

0.540 

Participated  in  group  job  searchb 

40.3 

36.8 

3.4 

0.263 

Participated  in  an  education/training  activity 

ABE/GED 

16.7 

14.9 

1.7 

0.436 

ESL 

4.1 

3.3 

0.9 

0.468 

College  courses 

14.0 

14.5 

-0.5 

0.825 

Vocational  training 

17.4 

18.6 

-1.2 

0.617 

During  past  vear 

Ever  participated  in  any  activity3  (%) 

34.7 

36.1 

-1.4 

0.647 

Participated  in  group  job  searchb  (%) 

18.6 

20.0 

-1.4 

0.576 

Participated  in  an  education/training  activity  (%) 

ABE/GED 

10.9 

9.6 

1.3 

0.495 

ESL 

1.7 

2.2 

-0.5 

0.552 

College  courses 

9.1 

8.5 

0.7 

0.701 

Vocational  training 

9.3 

9.9 

-0.6 

0.762 

Average  number  of  weeks  participating  in 

Group  job  search 

1.2 

1.2 

0.0 

0.905 

Education/training  activities 

3.7 

3.5 

0.2 

0.731 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aSample  members  are  counted  as  participating  in  any  activity  if  they  participated  in  a job  club,  ESL  training, 
high  school/GED/ABE  enrollment,  college  enrollment,  or  vocational  training. 
bGroup  job  search  is  also  known  as  a "job  club." 
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The  Employment  Retention  and  Advancement  Project 

Appendix  Table  D.2 

Impacts  on  Educational  Attainment 
at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Since  random  assignment 

Received  any  type  of  diploma. 

degree,  license,  or  certificate  (%) 

24.9 

19.9 

5.0  * 

0.058 

Received  a GED  certificate  (%) 

2.5 

3.0 

-0.4 

0.668 

Received  a high  school  diploma  (%) 

1.8 

1.8 

0.0 

0.983 

Received  a bachelor's  or  graduate  degree  (%) 

0.8 

0.2 

0.6 

0.173 

Received  a trade  license  or  trade  certificate  (%) 

21.3 

16.3 

5.0  ** 

0.042 

Number  of  trade  licenses  or  trade  certificates  received 

0.7 

0.7 

0.0 

0.973 

Type  of  trade  license  received  (%) 

CN  A/nursing 

4.4 

3.5 

0.9 

0.489 

Child  care 

2.5 

1.2 

1.3 

0.121 

Food  preparation 

2.8 

1.6 

1.2 

0.200 

Home  care 

1.5 

2.7 

-1.2 

0.184 

Janitor/housekeeping 

0.6 

0.8 

-0.2 

0.745 

Security 

3.5 

1.0 

9 5 *** 

0.008 

Cosmetics 

0.6 

0.2 

0.4 

0.339 

Computers 

0.8 

1.2 

-0.4 

0.495 

Other 

3.5 

1.8 

1.8  * 

0.083 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.3 

Impacts  on  Characteristics  of  Current  Job  at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Characteristics  of  current  iob 

Working  status  (%) 

Full  time 

55.8 

53.7 

2.1 

0.512 

Part  time 

10.7 

14.1 

-3.4 

0.106 

Seasonal,  temporary,  or  odd  job  (%) 

12.7 

15.7 

-3.0 

0.175 

Out-of-state  job  (%) 

1.5 

1.0 

0.5 

0.474 

Hours 

Average  hours  worked  per  week 

23.9 

23.1 

0.7 

0.525 

Average  days  worked  per  week 

3.3 

3.3 

0.0 

0.846 

Earninss 

Average  hourly  pay  ($) 

5.42 

5.30 

0.12 

0.707 

Average  hourly  wage  among  those  employed  ($) 

8.42 

7.92 

0.49 

NA 

Average  weekly  earnings  ($) 

193 

178 

15 

0.213 

Total  earnings  per  week  (%) 

Less  than  $200 

16.6 

21.8 

-5.2  ** 

0.046 

$201-$300 

20.5 

22.9 

-2.4 

0.386 

$301-$500 

21.7 

18.7 

3.0 

0.252 

$500  or  more 

5.4 

3.5 

2.0 

0.150 

Average  weekly  earnings  among  those  employed  ($) 

299 

266 

33 

NA 

Health  insurance  benefits  (%) 

Employer  offered  a health  plan  or  medical  insurance 

22.6 

19.5 

3.1 

0.221 

Enrolled  in  employer's  health  insurance  plan 

7.5 

6.2 

1.4 

0.391 

Reasons  why  not  enrolled  in  employer's  health  insurance  plan 

Covered  by  Medicaid 

5.4 

5.0 

0.4 

0.788 

Covered  by  other  health  insurance 

0.6 

0.4 

0.2 

0.645 

Too  expensive 

5.9 

4.7 

1.2 

0.410 

Had  not  worked  long  enough 

2.4 

2.6 

-0.3 

0.766 

Other  reason 

0.8 

0.2 

0.6 

0.184 

Sample  size  (total  = 1,023) 

521 

502 

(continued) 
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Appendix  Table  D.3  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

NA  = not  available.  In  this  case,  the  data  for  these  measures  were  not  collected. 

The  outcome  measures  are  discussed  in  Chapter  2. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.4 

Impacts  on  Type  of  Industry,  Type  of  Occupation,  and  Firm  Size  in  Current  Job 

at  the  42-Month  Survey: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Tvpe  of  industry  (%) 

Construction 

0.4 

0.2 

0.2 

0.547 

Manufacturing 

1.1 

1.7 

-0.5 

0.484 

Transportation  and  utilities 

4.3 

2.8 

1.6 

0.187 

Wholesale  trade 

0.0 

0.0 

0.0 

0.000 

Retail  trade 

8.6 

8.9 

-0.4 

0.840 

Finance,  insurance,  and  real  estate 

1.5 

2.3 

-0.8 

0.374 

Services 

48.2 

49.2 

-1.0 

0.755 

Other  industries 

2.1 

2.6 

-0.5 

0.601 

TvDe  of  occupation  ( % ) 

Sales 

10.0 

10.7 

-0.6 

0.744 

Clerical 

7.1 

8.9 

-1.7 

0.317 

Services 

35.9 

34.8 

1.1 

0.726 

Operatives/laborers 

9.7 

8.4 

1.3 

0.457 

Other 

3.8 

4.7 

-0.8 

0.511 

Number  of  workers  (%) 

1-4 

15.3 

14.1 

1.2 

0.581 

5-49 

22.8 

24.0 

-1.2 

0.674 

50-99 

5.2 

4.1 

1.1 

0.426 

100-249 

6.2 

7.8 

-1.6 

0.341 

250-499 

4.9 

5.7 

-0.8 

0.581 

500  or  more 

11.1 

11.9 

-0.7 

0.731 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.5 

Impacts  on  Employment  Retention  at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Current  iob 

Currently  employed  (%) 

66.6 

67.7 

-1.1 

0.706 

Average  months  employed  in  current  job 

19.8 

19.4 

0.4 

0.759 

Total  months  employed  in  current  job  (%) 

0 

34.4 

33.1 

1.3 

0.668 

1 to  6 

8.0 

11.3 

-3.2  * 

0.084 

7 to  12 

6.2 

6.0 

0.2 

0.918 

12  to  24 

10.4 

9.4 

1.1 

0.579 

More  than  24 

41.0 

40.3 

0.8 

0.806 

Prior  vear  (%) 

Employed  in  year  prior  to  interview 

79.1 

79.6 

-0.5 

0.850 

Total  months  employed  in  prior  year 

0 

14.5 

16.6 

-2.1 

0.346 

1 to  6 

18.1 

16.7 

1.4 

0.561 

7 to  11 

12.7 

12.1 

0.7 

0.750 

12 

54.7 

54.6 

0.1 

0.977 

Since  random  assignment  t%) 

Ever  employed  since  random  assignment 

92.9 

89.6 

3.3  * 

0.063 

Number  of  jobs  since  random  assignment 

0 

7.2 

10.4 

-3.2  * 

0.067 

1 

41.5 

41.3 

0.2 

0.942 

2 or  3 

41.2 

39.9 

1.3 

0.683 

4 to  6 

9.6 

7.6 

2.0 

0.260 

More  than  6 

0.6 

0.8 

-0.2 

0.645 

Ever  worked  for  one  employer  for  6 months 

or  more 

58.5 

53.5 

5.0 

0.108 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.6 

Impacts  on  Household  Income  and  Savings  at  the  42-Month  Survey: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Household  income 

Percentage  with  each  income  source 

Own  earnings 

67.2 

69.9 

-2.8 

0.345 

Earnings  of  other  members 

12.8 

12.7 

0.1 

0.981 

Child  support 

11.4 

10.7 

0.7 

0.730 

Public  assistance 

83.8 

80.6 

3.1 

0.183 

Cash  assistance 

12.0 

17.1 

_5  i ** 

0.021 

Food  stamps 

80.8 

79.2 

1.7 

0.503 

SSI  or  disability  benefits 

17.9 

18.1 

-0.3 

0.907 

Total  household  income  in  prior  month11  ($) 

1,301 

1,269 

33 

0.560 

Household  income 

$0  in  the  prior  month 

0.4 

1.7 

-1.2  * 

0.061 

$l-$  1,000 

45.9 

41.8 

4.0 

0.200 

$1,001 -$2,000 

39.4 

45.4 

-6.1  * 

0.055 

$2,001-$3,000 

11.3 

8.7 

2.5 

0.192 

More  than  $3,000 

3.1 

2.3 

0.7 

0.489 

Percentage  of  household  income  that  is  respondent's 

84.5 

82.1 

2.3 

0.203 

Below  poverty  levelb  (%) 

80.2 

82.6 

-2.4 

0.332 

Household  savings 

Total  household  savings  in  prior  month  ($) 

140 

83 

58 

0.382 

Distribution  of  household  savings  (%) 

$0  in  the  prior  month 

95.0 

93.8 

1.3 

0.396 

$l-$250 

1.2 

2.1 

-0.9 

0.250 

$251-$500 

0.8 

1.1 

-0.3 

0.646 

More  than  $501 

3.0 

3.1 

0.0 

0.969 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aTotal  household  income  is  defined  as  the  sum  of  all  sources  of  income  from  all  household  members, 
including  earnings,  public  assistance,  child  support,  and  any  other  sources  of  income. 

bThe  2005  U.S.  Census  Bureau  poverty  thresholds  (which  are  based  on  household  income,  household  size, 
and  number  of  related  minor  children)  were  used  to  determine  whether  sample  members  were  living  at  or  below 
the  poverty  level. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.7 

Impacts  on  Household  Composition,  Housing,  and  Housing  Expenditures 

at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Household  composition 

Number  of  people  in  household 

4.9 

4.8 

0.0 

0.879 

New  child  in  household  since  random  assignment  (%) 

17.5 

17.6 

-0.2 

0.950 

Marital  status 

Married  or  cohabitating  (%) 

Married  and  living  with  spouse 

3.7 

3.3 

0.4 

0.755 

Cohabitating  with  partner 

10.1 

6.8 

3.3  * 

0.060 

Single  (%) 

Divorced  or  separated 

Divorced 

6.1 

8.6 

-2.5 

0.128 

Separated 

8.3 

9.9 

-1.7 

0.356 

Never  married 

69.9 

69.8 

0.2 

0.958 

Widowed 

1.9 

1.2 

0.6 

0.410 

Housing 

Current  housing  status  (%) 

Owns  home 

2.9 

2.4 

0.5 

0.608 

Rents  home  or  apartment 

83.8 

80.2 

3.5 

0.139 

Lives  with  family  or  friend 

11.1 

15.6 

-4.5  ** 

0.034 

Lives  in  some  other  arrangement 

2.3 

1.9 

0.4 

0.644 

Household  expenditures 

Total  housing  expenditures  in  rent  or  mortgage  ($) 

546 

523 

23 

0.329 

Housing  cost  as  percentage  of  household  income  (%) 

45.6 

44.1 

1.4 

0.461 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.8 

Impacts  on  Health  and  Health  Care  Coverage  at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Health 

Self-rated  health  (%) 

Excellent 

20.8 

19.8 

1.0 

0.683 

Very  good 

20.3 

20.1 

0.2 

0.929 

Good 

35.0 

32.2 

2.8 

0.343 

Fair 

18.6 

21.0 

-2.4 

0.346 

Poor 

5.3 

7.0 

-1.7 

0.255 

Respondent  or  family  member  has  serious 

health  problems  that  limit  activities  or  work  (%) 

16.5 

19.2 

-2.7 

0.267 

Psychologicial  Distress  Scale3  (K6) 

5.7 

5.7 

0.0 

0.908 

Experienced  serious  psychological 

distress  in  the  past  month3  (%) 

11.0 

11.3 

-0.3 

0.877 

Health  care  coverage 

Respondent  has  health  care  coverage 

89.3 

89.4 

-0.1 

0.953 

Publicly  funded 

84.4 

86.0 

-1.7 

0.456 

Privately  funded 

10.4 

8.6 

1.9 

0.306 

All  dependent  children  have  health  care  coverage 

89.4 

87.1 

2.3 

0.228 

Respondent  and  all  children  have  health  care  coverage 

85.1 

83.8 

1.3 

0.555 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  group  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 

Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Based  on  the  K6  scale  that  includes  six  questions  about  how  often  a respondent  experienced  symptoms  of 
psychological  distress  during  the  past  30  days.  The  response  codes  (0-4)  of  the  six  items  for  each  person  are 
summed  to  yield  a scale  with  a 0-24  range.  A value  of  1 3 or  more  for  this  scale  is  used  here  to  define  serious 
psychological  distress  (Web  site:  http://www.hcp.med.harvard.edu/ncs/k6_scales.php). 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.9 

Impacts  on  Child  Care  and  Transportation  at  the  42-Month  Survey: 

Chicago 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P-Value 

Child  care 

During  past  month 

Any  child  care  (%) 

29.8 

26.8 

3.0 

0.260 

Any  informal  child  carea  (%) 

2.7 

2.9 

-0.2 

0.854 

Child  care  expenses  (%) 

25.4 

22.8 

2.6 

0.315 

Paid  entirely  by  respondent 

2.4 

4.3 

-1.9  * 

0.098 

Paid  partially  by  respondent 

18.6 

13.6 

5.0  ** 

0.028 

Not  paid  by  respondent 

4.4 

4.8 

-0.4 

0.783 

Child  care  expenses  ($) 

Paid  by  respondent 

28 

22 

7 

0.211 

Paid  by  someone  else 

39 

34 

5 

0.557 

Working  with  child  care  arrangement  (%) 

27.3 

24.0 

3.3 

0.218 

Working  without  child  care  arrangement  (%) 

36.3 

41.3 

-5.0  * 

0.098 

During  past  year 

Child  care  was  a barrier  to  school,  job  training,  or  work  (%) 
Respondent  quit  or  refused  job,  school,  or  training 

9.9 

12.9 

-3.0 

0.136 

because  of  child  care  problems  (%) 

4.7 

6.3 

-1.6 

0.251 

Respondent  missed  work  because  of  child  care  problems  (%) 

1.3 

2.6 

-1.3 

0.151 

Transportation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

33.4 

32.3 

1.1 

0.706 

0-15 

14.8 

19.4 

-4.5  * 

0.052 

16-30 

19.7 

18.1 

1.6 

0.517 

31-45 

13.5 

11.5 

1.9 

0.355 

46  or  more 

18.6 

18.7 

-0.1 

0.970 

Respondent  has  valid  driver's  license  (%) 

40.7 

44.1 

-3.5 

0.247 

Respondent  has  access  to  a car  (%) 

20.8 

22.2 

-1.4 

0.578 

Method  of  transportation  to  current  jobb  (%) 

By  car 

13.0 

16.3 

-3.3 

0.131 

By  bus  or  other  transportation 

40.8 

38.6 

2.2 

0.470 

Gets  a ride 

16.1 

12.9 

3.1 

0.158 

Walks 

5.4 

4.9 

0.5 

0.724 

During  past  month,  missed  work  due 

to  problems  with  transportation  (%) 

1.9 

2.1 

-0.2 

0.804 

Sample  size  (total  = 1,023) 

521 

502 

(continued) 
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Appendix  Table  D.9  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Informal  child  care  is  defined  as  a regular  child  care  arrangement  for  which  neither  the  respondent  nor  any 
other  person  or  institution  paid. 

bRespondents  can  select  more  than  one  method  of  transportation. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.10 

Impacts  on  Child  Care  for  Children  Ages  5 to  8 at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Tvne  of  child  care  arrangement  in  oast  month 

Currently  in  any  child  care  (%) 

57.3 

44.3 

13.0  ** 

0.013 

Number  of  child  care  arrangements 

0.6 

0.5 

q 2 *** 

0.008 

Any  home -based  care  (%) 

50.3 

40.4 

9.9  * 

0.060 

Family  daycare 

3.0 

5.9 

-2.9 

0.183 

Other  biological  parent 

1.8 

1.7 

0.1 

0.947 

Sibling 

7.5 

3.1 

4.3  * 

0.081 

Other  relative 

32.2 

24.1 

8.0  * 

0.098 

Spouse/partner 

0.4 

0.8 

-0.4 

0.676 

Nonrelative  in  child's  home 

8.2 

6.7 

1.5 

0.598 

Any  center-based  care  (%) 

7.8 

3.4 

4.4  * 

0.080 

Head  Start 

0.0 

0.0 

0.0 

0.000 

Preschool,  nursery,  or  daycare  center 

6.2 

2.8 

3.4 

0.128 

After-school  program 

1.1 

0.0 

1.1 

0.173 

Summer  camp 

0.5 

0.7 

-0.2 

0.862 

Extent  of  child  care  in  a tvnical  week 

Number  of  hours  in  child  care 

14 

10 

0.026 

0 hours  in  child  care  (%) 

44.1 

56.4 

-12.4  ** 

0.020 

Fewer  than  20  hours  in  child  care  (%) 

22.1 

17.9 

4.3 

0.330 

20  or  more  hours  in  child  care  (%) 

33.8 

25.7 

8.1 

0.103 

Sample  size  (total  = 353) 

189 

164 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by 
MDRC  at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.ll 

Impacts  on  School  Outcomes  and  Behavioral  Problems  for  Children  Ages  5 to  8 

at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

School  outcomes*  (%) 

School  performance 

Above  average 

63.3 

74.0 

-10.6  ** 

0.037 

Below  average 

7.8 

8.1 

-0.4 

0.896 

Ever  in  special  education 

9.6 

6.6 

3.0 

0.321 

Ever  repeated  a grade 

6.3 

8.6 

-2.3 

0.429 

Ever  suspended 

8.7 

9.6 

-0.9 

0.785 

Ever  expelled 

3.4 

4.1 

-0.7 

0.745 

Behavioral  problems*  (%) 

Child  has  behavior  that  is  hard  to  control 

10.6 

9.2 

1.4 

0.677 

Sample  size  (total  = 353) 

189 

164 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC 
at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.12 

Impacts  on  School  Outcomes  for  Children  Ages  13  to  17  at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

School  outcomes"  (%) 

School  performance 
Above  average 

47.8 

49.7 

-1.9 

0.686 

Below  average 

16.8 

11.7 

5.1 

0.131 

Ever  in  special  education 

20.7 

12.8 

y 9 ** 

0.028 

Ever  repeated  a grade 

18.9 

20.9 

-2.0 

0.597 

Ever  suspended 

29.7 

36.5 

-6.8 

0.132 

Ever  expelled 

7.6 

7.8 

-0.2 

0.934 

Ever  dropped  out  of  school 

6.8 

5.9 

1.0 

0.682 

Sample  size  (total  = 443) 

221 

222 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC 
at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  group  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.13 

Impacts  on  Behavioral  Problems  and  Police  Involvement  Outcomes 
for  Children  Ages  13  to  17  at  the  42-Month  Survey: 


Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Behavioral  problems3 1%) 

Child  has  behavior  that  is  hard  to  control 

8.3 

5.7 

2.6 

0.288 

Ever  had  a problem  with  alcohol  or  drugs 

1.8 

1.0 

0.8 

0.468 

Police  involvement  outcomes3  f%) 

Ever  in  juvenile  court 

10.0 

9.5 

0.6 

0.841 

Ever  in  trouble  with  the  police 

6.8 

8.2 

-1.5 

0.570 

Sample  size  (total  = 443) 

221 

222 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC  at 
the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.14 

Impacts  on  Participation  in  Job  Search,  Education,  and  Training  at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Since  random  assignment"  1%) 

Ever  participated  in  any  activity 

66.7 

59.2 

7 5** 

0.015 

Participated  in  group  job  searchb 

42.0 

40.3 

1.8 

0.574 

Participated  in  an  education/training  activity 

ABE/GED 

11.9 

9.5 

2.4 

0.228 

ESL 

5.2 

3.7 

1.5 

0.247 

College  courses 

27.7 

22.8 

4.9  * 

0.070 

Vocational  training 

26.1 

22.5 

3.6 

0.194 

During  past  year 

Ever  participated  in  any  activity"  (%) 

39.2 

35.7 

3.4 

0.267 

Participated  in  group  job  searchb  (%) 

17.2 

16.9 

0.3 

0.913 

Participated  in  an  education/training  activity  (%) 

ABE/GED 

5.4 

6.7 

-1.3 

0.387 

ESL 

2.7 

2.6 

0.1 

0.899 

College  courses 

17.0 

13.5 

3.5 

0.132 

Vocational  training 

11.6 

10.4 

1.2 

0.552 

Average  number  of  weeks  participating  in 

Group  job  searcn 

0.6 

0.6 

0.0 

0.893 

Education/training  activities 

6.3 

5.6 

0.7 

0.457 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  disucssed  in  Chapter  2. 

“Sample  members  are  counted  as  participating  in  any  activity  if  they  participated  in  a job  club,  ESL  training, 
high  school/GED/ABE  enrollment,  college  enrollment,  or  vocational  training. 
bGroup  job  search  is  also  known  as  a "job  club." 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.15 

Impacts  on  Educational  Attainment  at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Since  random  assignment 

Received  any  type  of  diploma, 

degree,  license,  or  certificate  (%) 

24.8 

25.7 

-1.0 

0.729 

Received  a GED  certificate  (%) 

1.3 

2.5 

-1.2 

0.164 

Received  a high  school  diploma  (%) 

1.6 

1.7 

-0.1 

0.946 

Received  a bachelor's  or  graduate  degree  (%) 

0.5 

0.6 

-0.1 

0.791 

Received  a trade  license  or  trade  certificate  (%) 

21.3 

21.1 

0.2 

0.955 

Number  of  trade  licenses  or  trade  certificates  received 

0.8 

0.8 

0.0 

0.886 

Type  of  trade  license  received  (%) 

CNA/nursing 

5.8 

4.1 

1.6 

0.246 

Child  care 

1.9 

1.2 

0.8 

0.344 

Food  preparation 

0.2 

0.7 

-0.5 

0.222 

Florne  care 

0.2 

0.0 

0.2 

0.267 

Janitor/housekeeping 

0.2 

0.0 

0.2 

0.302 

Security 

1.0 

1.0 

0.0 

0.967 

Cosmetics 

0.4 

0.4 

0.1 

0.876 

Computers 

1.6 

1.5 

0.2 

0.851 

Other 

5.6 

7.4 

-1.8 

0.275 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.16 

Impacts  on  Characteristics  of  Current  Job  at  the  42-Month  Survey: 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Characteristics  of  current  iob 

Working  status  (%) 

Full  time 

51.8 

54.7 

-3.0 

0.349 

Part  time 

8.0 

4.1 

39** 

0.012 

Seasonal,  temporary,  or  odd  job  (%) 

14.0 

12.1 

1.9 

0.386 

Out-of-state  job  (%) 

1.8 

1.3 

0.5 

0.503 

Hours 

Average  hours  worked  per  week 

22.4 

23.2 

-0.8 

0.515 

Average  days  worked  per  week 

2.9 

3.0 

0.0 

0.774 

Famines 

Average  hourly  pay  ($) 

6.40 

6.25 

0.15 

0.719 

Average  hourly  wage  among  those  employed  ($) 

10.90 

10.72 

0.18 

NA 

Average  weekly  earnings  ($) 

240 

251 

-11 

0.514 

Total  earnings  per  week  (%) 

Less  than  200 

6.9 

3.8 

3.1  ** 

0.046 

$201-$300 

12.5 

10.0 

2.5 

0.250 

$301-$500 

24.5 

29.1 

-4.6 

0.119 

$500  or  more 

15.0 

15.7 

-0.7 

0.764 

Average  weekly  earnings  among  those  employed  ($) 

410 

428 

-18 

NA 

Health  insurance  benefits  1%) 

Employer  offered  a health  plan  or  medical  insurance 

37.2 

35.9 

1.2 

0.690 

Enrolled  in  employer's  health  insurance  plan 

16.8 

16.0 

0.8 

0.725 

Reasons  why  not  enrolled  in  employer's  health  insurance  plan 

Covered  by  Medicaid 

7.6 

7.5 

0.1 

0.967 

Covered  by  other  health  insurance 

2.1 

1.7 

0.4 

0.651 

Too  expensive 

6.0 

7.4 

-1.4 

0.388 

Had  not  worked  long  enough 

3.3 

1.9 

1.5 

0.162 

Other  reason 

0.6 

0.7 

-0.1 

0.829 

Sample  size  (total  = 982) 

498 

484 

(continued) 
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Appendix  Table  D.16  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

NA  = not  available.  In  this  case,  the  data  for  these  measures  were  not  collected. 

The  outcome  measures  are  discussed  in  Chapter  2. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.17 

Impacts  on  Type  of  Industry,  Type  of  Occupation,  and  Firm  Size  in  Current  Job 

at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Tvpe  of  industry  (%) 

Construction 

0.1 

2.0 

_ ] g *** 

0.004 

Manufacturing 

4.6 

5.3 

-0.7 

0.633 

Transportation  and  utilities 

7.7 

4.7 

2.9  * 

0.066 

Wholesale  trade 

0.7 

0.4 

0.3 

0.582 

Retail  trade 

7.0 

6.1 

0.9 

0.603 

Finance,  insurance,  and  real  estate 

3.2 

3.3 

-0.1 

0.904 

Services 

33.0 

33.5 

-0.5 

0.867 

Other  industries 

3.1 

3.5 

-0.4 

0.756 

TvDe  of  occupation  (%) 

Sales 

7.5 

5.0 

2.5 

0.114 

Clerical 

14.7 

17.5 

-2.8 

0.230 

Services 

21.6 

21.2 

0.5 

0.867 

Operatives/laborers 

11.5 

11.9 

-0.4 

0.841 

Other 

4.4 

3.4 

1.0 

0.415 

Number  of  workers  (%) 

1-4 

8.5 

7.0 

1.5 

0.397 

5-49 

18.4 

20.0 

-1.6 

0.544 

50-99 

6.0 

6.8 

-0.8 

0.629 

100-249 

10.5 

8.9 

1.5 

0.436 

250-499 

4.7 

5.0 

-0.3 

0.864 

500  or  more 

11.6 

12.1 

-0.5 

0.808 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statisitcal  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 


432 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.18 

Impacts  on  Employment  Retention  at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Current  job 

Currently  employed  (%) 

60.0 

59.2 

0.8 

0.791 

Average  months  employed  in  current  job 

14.3 

13.0 

1.3 

0.221 

Total  months  employed  in  current  job  (%) 

0 

41.5 

41.9 

-0.4 

0.902 

1 to  6 

11.0 

14.8 

-3.8  * 

0.083 

7 to  12 

7.5 

6.7 

0.9 

0.609 

12  to  24 

11.4 

9.5 

1.9 

0.349 

More  than  24 

28.6 

27.1 

1.4 

0.610 

Prior  vear  t%) 

Employed  in  year  prior  to  interview 

79.0 

78.5 

0.5 

0.836 

Total  months  employed  in  prior  year 

0 

14.7 

16.8 

-2.0 

0.377 

1 to  6 

26.0 

23.1 

2.9 

0.296 

7 to  11 

14.6 

16.7 

-2.1 

0.371 

12 

44.7 

43.4 

1.2 

0.692 

Random  assignment  (%) 

Ever  employed  since  random  assignment 

91.2 

89.0 

2.2 

0.244 

Number  of  jobs  since  random  assigment 

0 

8.9 

11.2 

-2.3 

0.227 

1 

29.7 

29.2 

0.5 

0.869 

2 or  3 

46.9 

43.3 

3.6 

0.262 

4 to  6 

13.6 

14.9 

-1.3 

0.557 

More  than  6 

1.0 

1.5 

-0.5 

0.509 

Ever  worked  for  one  employer  for  6 months  or  more 

60.4 

62.1 

-1.7 

0.587 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.19 

Impacts  on  Household  Income  and  Savings  at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Household  income 

Percentage  with  each  income  source 

Own  earnings 

60.4 

63.4 

-3.0 

0.335 

Earnings  of  other  members 

26.5 

31.3 

-4.8  * 

0.095 

Child  support 

12.2 

13.4 

-1.3 

0.567 

Public  assistance 

62.3 

58.6 

3.6 

0.238 

Cash  assistance 

36.5 

34.3 

2.2 

0.477 

Food  stamps 

56.4 

53.0 

3.4 

0.265 

SSI  or  disability  benefits 

12.5 

11.9 

0.7 

0.760 

Total  household  income  in  prior  month3  ($) 

1,710 

1,837 

-127 

0.129 

Household  income 

$0  in  the  prior  month 

0.9 

1.1 

-0.3 

0.693 

$l-$  1,000 

29.3 

29.1 

0.2 

0.947 

$1,001 -$2,000 

44.2 

39.2 

5.0 

0.126 

$2,001-$3,000 

15.9 

17.8 

-1.9 

0.438 

More  than  $3,000 

9.8 

12.8 

-3.0 

0.150 

Percentage  of  household  income  that  is  respondent's 

77.8 

76.8 

1.0 

0.629 

Below  poverty  levelb  (%) 

57.4 

54.6 

2.7 

0.403 

Household  savings 

Total  household  savings  in  prior  month  ($) 

221 

380 

-159 

0.119 

Distribution  of  household  savings  (%) 

$0  in  the  prior  month 

83.8 

83.6 

0.2 

0.929 

$l-$250 

7.5 

5.9 

1.7 

0.311 

$251-$500 

1.6 

2.9 

-1.3 

0.204 

$501  or  more 

7.0 

7.7 

-0.6 

0.715 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aTotal  household  income  is  defined  as  the  sum  of  all  sources  of  income  from  all  household  members, 
including  earnings,  public  assistance,  child  support,  and  any  other  sources  of  income. 

bThe  2005  U.S.  Census  Bureau  poverty  thresholds  (which  are  based  on  household  income,  household  size,  and 
number  of  related  minor  children)  were  used  to  determine  whether  sample  members  were  living  at  or  below  the 
poverty  level. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.20 

Impacts  on  Household  Composition,  Housing,  and  Housing  Expenditures 

at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Household  composition 

Number  of  people  in  household 

3.9 

4.0 

-0.1 

0.105 

New  child  in  household  since  random  assignment  (%) 

22.8 

26.3 

-3.5 

0.182 

Marital  status 

Married  or  cohabitating  (%) 

Married  and  living  with  spouse 

8.1 

7.7 

0.4 

0.808 

Cohabitating  with  partner 

9.5 

15.1 

-5  6 *** 

0.008 

Single  (%) 

Divorced  or  separated 

Divorced 

10.1 

9.7 

0.4 

0.823 

Separated 

12.3 

10.1 

2.2 

0.282 

Never  married 

57.9 

55.9 

2.0 

0.503 

Widowed 

1.8 

0.9 

1.0 

0.199 

Housing 

Current  housing  status  (%) 

Owns  home 

3.7 

2.7 

1.1 

0.354 

Rents  home  or  apartment 

66.4 

67.8 

-1.5 

0.615 

Lives  with  family  or  friend 

26.0 

27.7 

-1.8 

0.520 

Lives  in  some  other  arrangement 

4.0 

1.8 

2.2  ** 

0.044 

Household  exDenditures 

Total  housing  expenditures  in  rent  or  mortgage  ($) 

778 

806 

-28 

0.375 

Housing  cost  as  percentage  of  household  income  (%) 

51.2 

49.6 

1.6 

0.394 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.21 

Impacts  on  Health  and  Health  Care  Coverage  at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Health 

Self-rated  health  (%) 

Excellent 

19.5 

24.0 

-4.5  * 

0.090 

Very  good 

19.7 

19.5 

0.2 

0.933 

Good 

36.2 

34.5 

1.7 

0.597 

Fair 

18.9 

17.6 

1.3 

0.598 

Poor 

5.8 

4.5 

1.3 

0.351 

Respondent  or  family  member  has  serious 

health  problems  that  limit  activities  or  work  (%) 

18.3 

14.7 

3.6 

0.127 

Psychologicial  Distress  Scale3  (K6) 

5.7 

5.3 

0.4 

0.247 

Experienced  serious  psychological 

distress  in  the  past  month3  (%) 

12.4 

11.5 

0.9 

0.663 

Health  care  coverage 

Respondent  has  health  care  coverage 

86.4 

85.5 

0.9 

0.688 

Publicly  funded 

75.2 

72.8 

2.4 

0.393 

Privately  funded 

20.9 

20.4 

0.4 

0.866 

All  dependent  children  have  health  care  coverage 

82.6 

82.7 

-0.1 

0.982 

Respondent  and  all  children  have  health  care  coverage 

79.7 

77.8 

2.0 

0.456 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Based  on  the  K6  scale  that  includes  six  questions  about  how  often  a respondent  experienced  symptoms  of 
psychological  distress  during  the  past  30  days.  The  response  codes  (0-4)  of  the  six  items  for  each  person  are 
summed  to  yield  a scale  with  a 0-24  range.  A value  of  1 3 or  more  for  this  scale  is  used  here  to  define  serious 
psychological  distress  (Web  site:  http://www.hcp.med.harvard.edu/ncs/k6_scales.php). 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.22 

Impacts  on  Child  Care  and  Transportation  at  the  42-Month  Survey: 


Los  Angeles  RFS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P- Value 

Child  care 

During  past  month 

Any  child  care  (%) 

38.4 

35.7 

2.7 

0.364 

Any  informal  child  carea  (%) 

8.2 

5.3 

2.9  * 

0.082 

Child  care  expenses  (%) 

27.8 

27.7 

0.1 

0.984 

Paid  entirely  by  respondent 

8.8 

8.3 

0.5 

0.803 

Paid  partially  by  respondent 

5.4 

6.9 

-1.5 

0.330 

Not  paid  by  respondent 

13.6 

12.3 

1.2 

0.565 

Child  care  expenses  ($) 

Paid  by  respondent 

46 

48 

-2 

0.798 

Paid  by  someone  else 

30 

42 

-12 

0.248 

Working  with  child  care  arrangement  (%) 

30.1 

28.4 

1.8 

0.540 

Working  without  child  care  arrangement  (%) 

26.2 

29.1 

-2.8 

0.311 

During  past  year 

Child  care  was  a barrier  to  school,  job  training,  or  work  (%) 
Respondent  quit  or  refused  job,  school,  or  training 

25.4 

23.8 

1.6 

0.560 

because  of  child  care  problems  (%) 

15.3 

14.6 

0.7 

0.764 

Respondent  missed  work  because  of  child  care  problems  (%) 

5.3 

5.0 

0.3 

0.848 

Transportation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

40.0 

40.8 

-0.8 

0.791 

0-15 

24.3 

21.1 

3.3 

0.233 

16-30 

22.9 

22.7 

0.2 

0.949 

31-45 

7.1 

7.8 

-0.6 

0.708 

46  or  more 

5.5 

7.8 

-2.3 

0.160 

Respondent  has  valid  driver's  license  (%) 

75.2 

69.6 

5.6  * 

0.052 

Respondent  has  access  to  a car  (%) 

59.7 

54.0 

5.7  * 

0.069 

Method  of  transportation  to  current  jobb  (%) 

By  car 

40.8 

40.0 

0.7 

0.816 

By  bus  or  other  transportation 

14.4 

12.3 

2.2 

0.321 

Gets  a ride 

9.2 

10.5 

-1.3 

0.510 

Walks 

3.6 

1.9 

1.7 

0.119 

During  past  month,  missed  work  due 

to  problems  with  transportation  (%) 

1.9 

1.8 

0.0 

0.966 

Sample  size  (total  = 982) 

498 

484 

(continued) 
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Appendix  Table  D.22  (continued) 


SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Informal  child  care  is  defined  as  a regular  child  care  arrangement  for  which  neither  the  respondent  nor  any 
other  person  or  institution  paid. 

bRespondents  can  select  more  than  one  method  of  transportation. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.23 

Impacts  on  Child  Care  for  Children  Ages  5 to  8 at  the  42-Month  Survey: 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

TvDe  of  child  care  arrangement  in  nast  month 

Currently  any  child  care  (%) 

51.4 

52.2 

-0.8 

0.869 

Number  of  child  care  arrangements 

0.5 

0.6 

0.0 

0.725 

Any  home-based  care  (%) 

43.1 

44.9 

-1.8 

0.714 

Family  daycare 

8.8 

6.5 

2.2 

0.406 

Other  biological  parent 

0.4 

2.5 

-2.1  * 

0.073 

Sibling 

1.9 

3.8 

-2.0 

0.235 

Other  relative 

28.1 

28.6 

-0.5 

0.909 

Spouse/partner 

0.5 

1.4 

-0.9 

0.360 

Nonrelative  in  child's  home 

5.4 

3.2 

2.2 

0.289 

Any  center-based  care  (%) 

9.1 

8.6 

0.5 

0.865 

Eiead  Start 

0.5 

0.0 

0.5 

0.278 

Preschool,  nursery,  or  daycare  center 

4.2 

5.3 

-1.1 

0.608 

After-school  program 

3.8 

3.4 

0.4 

0.813 

Summer  camp 

0.5 

-0.1 

0.6 

0.240 

Extent  of  child  care  in  a tvDical  week 

Number  of  hours  in  child  care 

12 

12 

0 

0.967 

0 hours  in  child  care  (%) 

49.8 

49.7 

0.1 

0.982 

Fewer  than  20  hours  in  child  care  (%) 

19.8 

25.7 

-6.0 

0.167 

20  or  more  hours  in  child  care  (%) 

30.4 

24.6 

5.8 

0.203 

Sample  size  (total  = 422) 

206 

216 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by 
MDRC  at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.24 

Impacts  on  School  Outcomes  and  Behavioral  Problems  for  Children  Ages  5 to  8 

at  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

School  outcomes3  (%) 

School  performance 

Above  average 

71.4 

71.9 

-0.5 

0.904 

Below  average 

4.6 

7.6 

-3.0 

0.207 

Ever  in  special  education 

7.9 

7.8 

0.1 

0.968 

Ever  repeated  a grade 

4.1 

6.8 

-2.7 

0.232 

Ever  suspended 

3.9 

2.7 

1.2 

0.502 

Ever  expelled 

1.5 

0.4 

1.0 

0.286 

Behavioral  problems3  (%) 

Child  has  behavior  that  is  hard  to  control 

5.8 

6.1 

-0.3 

0.889 

Sample  size  (total  = 422) 

206 

216 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by 
MDRC  at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.25 

Impacts  on  School  Outcomes  for  Children  Ages  13  to  17  at  the  42-Month  Survey: 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

School  outcomes3  (%) 

School  performance 

Above  average 

52.1 

50.3 

1.8 

0.792 

Below  average 

10.9 

13.2 

-2.3 

0.587 

Ever  in  special  education 

12.3 

16.7 

-4.4 

0.360 

Ever  repeated  a grade 

9.8 

15.1 

-5.3 

0.236 

Ever  suspended 

31.3 

29.0 

2.3 

0.708 

Ever  expelled 

10.3 

7.0 

3.3 

0.393 

Ever  dropped  out  of  school 

1.4 

4.5 

-3.2 

0.145 

Sample  size  (total  = 243) 

126 

117 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC 
at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“As  reported  by  respondent. 


441 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.26 

Impacts  on  Behavioral  Problems  and  Police  Involvement  Outcomes 
for  Children  Ages  13  to  17  at  the  42-Month  Survey: 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Behavioral  problems3 1%) 

Child  has  behavior  that  is  hard  to  control 

11.1 

6.0 

5.2 

0.171 

Ever  had  a problem  with  alcohol  or  drugs 

0.7 

3.5 

-2.8 

0.145 

Police  involvement  outcomes3  (%) 

Ever  in  juvenile  court 

9.6 

11.1 

-1.4 

0.730 

Ever  in  trouble  with  the  police 

6.9 

4.6 

2.4 

0.464 

Sample  size  (total  = 243) 

126 

117 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC  at 
the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.27 

Impacts  on  Participation  in  Job  Search,  Education,  and  Training  at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Since  random  assignment  (%) 

Ever  participated  in  any  activity3 

62.2 

62.6 

-0.4 

0.906 

Participated  in  group  job  searchb 

35.4 

44.8 

4 *** 

0.006 

Participated  in  an  education/training  activity 

ABE/GED 

10.0 

8.0 

2.1 

0.298 

ESL 

5.0 

2.3 

9 ~j  ** 

0.044 

College  courses 

28.3 

24.2 

4.2 

0.160 

Vocational  training 

18.0 

21.5 

-3.5 

0.215 

During  past  vear 

Ever  participated  in  any  activity3  (%) 

35.5 

38.9 

-3.4 

0.314 

Participated  in  group  job  searchb  (%) 

13.3 

21.9 

-8.6  *** 

0.001 

Participated  in  an  education/training  activity  (%) 

ABE/GED 

5.0 

4.1 

0.8 

0.582 

ESL 

2.0 

0.8 

1.2 

0.143 

College  courses 

17.6 

14.6 

3.0 

0.232 

Vocational  training 

7.0 

11.1 

-4.1  ** 

0.040 

Average  number  of  weeks  participating  in 

Group  job  search 

0.3 

0.8 

-0  5 *** 

0.001 

Education/training  activities 

5.0 

6.3 

-1.3 

0.162 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Sample  members  are  counted  as  participating  in  any  activity  if  they  participated  in  a job  club,  ESL  training, 
high  school/GED/ABE  enrollment,  college  enrollment,  or  vocational  training. 
bGroup  job  search  is  also  known  as  a "job  club." 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.28 

Impacts  on  Educational  Attainment  at  the  42-Month  Survey: 
Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Since  random  assignment 

Received  any  type  of  diploma, 

degree,  license,  or  certificate  (%) 

20.6 

24.6 

-4.0 

0.163 

Received  a GED  certificate 

1.7 

1.2 

0.5 

0.567 

Received  a high  school  diploma 

1.1 

0.7 

0.5 

0.503 

Received  a bachelor's  or  graduate  degree 

1.1 

2.1 

-1.0 

0.258 

Received  a trade  license  or  trade  certificate 

16.9 

20.5 

-3.6 

0.180 

Number  of  trade  licenses  or  trade  certificates  received 

0.7 

0.8 

-0.1 

0.133 

Type  of  trade  license  received  (%) 

CNA/nursing 

4.2 

4.4 

-0.2 

0.913 

Child  care 

0.8 

1.5 

-0.7 

0.339 

Food  preparation 

0.0 

0.5 

-0.5 

0.117 

Elome  care 

0.0 

0.5 

-0.5 

0.138 

J anitor/housekeeping 

0.0 

0.0 

0.0 

0.000 

Security 

1.2 

0.8 

0.4 

0.561 

Cosmetics 

0.5 

0.4 

0.0 

0.933 

Computers 

1.2 

1.4 

-0.2 

0.769 

Other 

6.4 

6.0 

0.4 

0.836 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.29 

Impacts  on  Characteristics  of  Current  Job  at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Characteristics  of  current  iob 

Working  status  (%) 

Full  time 

52.5 

52.1 

0.4 

0.902 

Part  time 

9.8 

9.9 

0.0 

0.989 

Seasonal,  temporary,  or  odd  job  (%) 

9.8 

10.0 

-0.2 

0.926 

Out-of-state  job  (%) 

3.6 

3.9 

-0.4 

0.791 

Hours 

Average  hours  worked  per  week 

23.7 

22.8 

1.0 

0.487 

Average  days  worked  per  week 

3.1 

3.0 

0.1 

0.681 

Famines 

Average  hourly  pay  ($) 

6.91 

6.60 

0.31 

0.485 

Average  hourly  wage  among  those  employed  ($) 

11.07 

10.87 

0.20 

N/A 

Average  weekly  earnings  ($) 

261 

241 

21 

0.247 

Total  earnings  per  week  (%) 

Less  than  200 

6.8 

8.8 

-2.0 

0.308 

$201-$300 

11.3 

13.3 

-2.0 

0.407 

$301-$500 

29.5 

23.9 

5.6  * 

0.087 

$500  or  more 

14.8 

15.3 

-0.5 

0.839 

Average  weekly  earnings  among  those  employed  ($) 

417 

398 

19 

NA 

Health  insurance  benefits  (%) 

Employer  offered  a health  plan  or  medical  insurance 

38.1 

40.4 

-2.3 

0.495 

Enrolled  in  employer's  health  insurance  plan 

19.5 

17.7 

1.9 

0.493 

Reasons  why  not  enrolled  in  employer's  health  insurance  plan 

Covered  by  Medicaid 

7.1 

7.4 

-0.3 

0.867 

Covered  by  other  health  insurance 

2.5 

2.3 

0.2 

0.838 

Too  expensive 

5.2 

8.0 

-2.8 

0.109 

Had  not  worked  long  enough 

2.8 

4.1 

-1.3 

0.301 

Other  reason 

0.4 

0.8 

-0.4 

0.433 

Sample  size  (total  = 892) 

504 

388 

(continued) 


445 


Appendix  Table  D.29  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

NA  = not  available.  In  this  case,  the  data  for  these  measures  were  not  collected. 

The  outcome  measures  are  discussed  in  Chapter  2. 

Italic  type  indicates  comparisons  that  are  nonexperimental.  These  measures  were  computed  only  for  sample 
members  who  were  employed.  Since  there  may  be  differences  in  the  characteristics  of  program  group  and  control 
group  members  who  were  employed,  any  differences  in  outcomes  may  not  necessarily  be  attributable  to  the  ERA 
program.  Statistical  tests  were  not  performed. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.30 

Impacts  on  Type  of  Industry,  Type  of  Occupation,  and  Firm  Size  in  Current  Job 

at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Tvpe  of  industry  (%) 

Construction 

1.9 

3.8 

-1.9 

0.103 

Manufacturing 

4.8 

2.7 

2.0 

0.137 

Transportation  and  utilities 

4.6 

3.2 

1.4 

0.321 

Wholesale  trade 

0.2 

0.2 

0.0 

0.924 

Retail  trade 

9.5 

oo 
o 6 

0.8 

0.716 

Finance,  insurance,  and  real  estate 

2.9 

2.5 

0.4 

0.750 

Services 

35.5 

36.2 

-0.7 

0.831 

Other  industries 

3.0 

4.0 

-1.1 

0.409 

TvDe  of  occupation  (%) 

Sales 

9.2 

9.5 

-0.3 

0.896 

Clerical 

13.9 

12.3 

1.5 

0.520 

Services 

21.4 

22.8 

-1.4 

0.623 

Operatives/laborers 

13.1 

10.7 

2.4 

0.271 

Other 

5.0 

6.5 

-1.6 

0.339 

Number  of  workers  (%) 

1-4 

8.9 

7.4 

1.5 

0.436 

5-49 

23.9 

20.5 

3.4 

0.252 

50-99 

6.0 

7.7 

-1.7 

0.344 

100-249 

6.9 

7.5 

-0.6 

0.764 

250-499 

4.3 

5.8 

-1.5 

0.344 

500  or  more 

13.0 

13.1 

-0.1 

0.981 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.31 

Impacts  on  Employment  Retention  at  the  42-Month  Survey: 
Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Current  iob 

Currently  employed  (%) 

62.5 

62.0 

0.6 

0.860 

Average  months  employed  in  current  job 

13.9 

14.0 

-0.1 

0.929 

Total  months  employed  in  current  job  (%) 

0 

39.7 

40.9 

-1.2 

0.722 

1 to  6 

13.4 

15.7 

-2.3 

0.354 

7 to  12 

10.6 

6.9 

3.7  * 

0.068 

12  to  24 

11.2 

10.0 

1.2 

0.583 

More  than  24 

25.1 

26.5 

-1.4 

0.641 

Prior  vear  (%) 

Employed  in  year  prior  to  interview 

84.4 

82.8 

1.6 

0.524 

Total  months  employed  in  prior  year 

0 

9.0 

11.3 

-2.2 

0.290 

1 to  6 

25.5 

24.3 

1.3 

0.674 

7 to  11 

19.0 

22.9 

-4.0 

0.163 

12 

46.5 

41.5 

4.9 

0.144 

Since  random  assignment  (%) 

Ever  employed  since  random  assignment 

96.1 

96.3 

-0.1 

0.923 

Number  of  jobs  since  random  assigment 

0 

4.0 

3.8 

0.2 

0.901 

1 

24.6 

28.2 

-3.6 

0.238 

2 or  3 

51.0 

43.1 

79** 

0.026 

4 to  6 

18.4 

21.6 

-3.2 

0.260 

More  than  6 

2.1 

3.4 

-1.3 

0.248 

Ever  worked  for  one  employer  for  6 months  or  more 

71.2 

70.5 

0.7 

0.828 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.32 

Impacts  on  Household  Income  and  Savings  at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Household  income 

Percentage  with  each  income  source 

Own  earnings 

67.1 

63.4 

3.7 

0.259 

Earnings  of  other  members 

40.2 

39.0 

1.2 

0.725 

Child  support 

16.1 

17.2 

-1.1 

0.673 

Public  assistance 

51.2 

55.6 

-4.4 

0.195 

Cash  assistance 

25.2 

28.9 

-3.7 

0.218 

Food  stamps 

39.9 

42.1 

-2.2 

0.504 

SSI  or  disability  benefits 

18.3 

18.1 

0.2 

0.950 

Total  household  income  in  prior  month3  ($) 

2,036 

2,074 

-38 

0.700 

Household  income 

1.3 

2.2 

-0.9 

0.327 

$0  in  the  prior  month 

$l-$  1,000 

22.9 

22.0 

0.9 

0.773 

$1,001 -$2,000 

36.5 

39.9 

-3.5 

0.328 

$2,001-$3,000 

21.4 

19.5 

2.0 

0.509 

More  than  $3,000 

18.0 

16.4 

1.6 

0.569 

Percentage  of  household  income  that  is  respondent's 

71.5 

67.4 

4.1 

0.101 

Below  poverty  levelb  (%) 

49.5 

48.4 

1.1 

0.754 

Household  savings 

Total  household  savings  in  prior  month  ($) 

265 

444 

-179 

0.493 

Distribution  of  household  savings  (%) 

$0  in  prior  month 

84.1 

83.5 

0.6 

0.832 

$l-$250 

8.3 

6.7 

1.6 

0.400 

$251-$500 

2.5 

1.8 

0.7 

0.512 

$501  or  more 

5.1 

8.0 

-2.9  * 

0.096 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aTotal  household  income  is  defined  as  the  sum  of  all  sources  of  income  from  all  household  members, 
including  earnings,  public  assistance,  child  support,  and  any  other  sources  of  income. 

bThe  2005  U.S.  Census  Bureau  poverty  thresholds  (which  are  based  on  household  income,  household  size,  and 
number  of  related  minor  children)  were  used  to  determine  whether  sample  members  were  living  at  or  below  the 
poverty  level. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.33 

Impacts  on  Household  Composition,  Housing,  and  Housing  Expenditures 

at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Household  composition 

Number  in  household 

4.3 

4.2 

0.1 

0.393 

New  child  in  household  since  random  assignment  (%) 

26.7 

29.0 

-2.3 

0.432 

Marital  status 

Married  or  cohabitating  (%) 

Married  and  living  with  spouse 

17.0 

16.1 

0.9 

0.722 

Cohabitating  with  partner 

16.7 

15.4 

1.3 

0.614 

Single  (%) 

Divorced  or  separated 

Divorced 

18.3 

14.1 

4.3  * 

0.093 

Separated 

13.9 

14.6 

-0.7 

0.790 

Never  married 

32.6 

37.1 

-4.5 

0.177 

Widowed 

1.2 

2.6 

-1.4 

0.143 

Housing 

Current  housing  status  (%) 

Owns  home 

9.2 

12.1 

-2.9 

0.185 

Rents  home  or  apartment 

57.4 

55.1 

2.4 

0.491 

Lives  with  family  or  friend 

28.6 

28.0 

0.6 

0.849 

Lives  in  some  other  arrangement 

4.7 

4.8 

-0.1 

0.940 

Household  expenditures 

Total  housing  expenditures  in  rent  or  mortgage  ($) 

735 

817 

-83  * 

0.060 

Housing  cost  as  percentage  of  household  income  (%) 

40.9 

43.0 

-2.0 

0.304 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.34 

Impacts  on  Health  and  Health  Care  Coverage  at  the  42-Month  Survey: 


Riverside  PASS 

Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Health 

Self-rated  health  (%) 

Excellent 

18.3 

15.5 

2.8 

0.290 

Very  good 

20.9 

19.9 

1.0 

0.730 

Good 

33.8 

37.0 

-3.2 

0.339 

Fair 

19.0 

19.7 

-0.7 

0.791 

Poor 

8.1 

7.9 

0.2 

0.929 

Respondent  or  family  member  has  serious 

health  problems  that  limit  activities  or  work  (%) 

20.1 

24.4 

-4.3 

0.132 

Psychologicial  Distress  Scale3  (K6) 

6.2 

6.6 

-0.4 

0.309 

Experienced  serious  psychological 

distress  in  the  past  month3  (%) 

14.0 

17.8 

-3.8 

0.141 

Health  care  coverage  (%) 

Respondent  has  health  care  coverage 

79.5 

81.4 

-2.0 

0.479 

Publicly  funded 

61.1 

66.8 

-5.7  * 

0.086 

Privately  funded 

25.2 

22.9 

2.3 

0.434 

All  dependent  children  have  health  care  coverage 

78.6 

77.1 

1.5 

0.596 

Respondent  and  all  children  have  health  care  coverage 

72.5 

70.5 

1.9 

0.534 

Sample  size  (total  = 892) 

504 

388 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose 
by  chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

“Based  on  the  K6  scale  that  includes  six  questions  about  how  often  a respondent  experienced  symptoms  of 
psychological  distress  during  the  past  30  days.  The  response  codes  (0-4)  of  the  six  items  for  each  person  are 
summed  to  yield  a scale  with  a 0-24  range.  A value  of  13  or  more  for  this  scale  is  used  here  to  define  serious 
psychological  distress  (Web  site:  http://www.hcp.med.harvard.edu/ncs/k6_scales.php). 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.35 

Impacts  on  Child  Care  and  Transportation  at  the  42-Month  Survey: 

Riverside  PASS 


ERA 

Control 

Difference 

Outcome 

Group 

Group 

(Impact) 

P -Value 

Child  care 

During  past  month 

Any  child  care  (%) 

35.5 

34.1 

1.5 

0.649 

Any  informal  child  carea  (%) 

8.1 

6.9 

1.1 

0.559 

Child  care  expenses  (%) 

26.0 

25.3 

0.7 

0.817 

Paid  entirely  by  respondent 

8.6 

8.1 

0.5 

0.784 

Paid  partially  by  respondent 

6.1 

6.6 

-0.5 

0.787 

Not  paid  by  respondent 

11.3 

10.7 

0.6 

0.774 

Child  care  expenses  ($) 

Paid  by  respondent 

42 

40 

2 

0.846 

Paid  by  someone  else 

43 

48 

-5 

0.711 

Working  with  child  care  arrangement  (%) 

31.2 

28.0 

3.2 

0.306 

Working  without  child  care  arrangement  (%) 

29.6 

28.2 

1.3 

0.667 

During  past  year 

Child  care  was  a barrier  to  school,  job  training,  or  work  (%) 
Respondent  quit  or  refused  job,  school,  or  training 

19.9 

25.2 

-5.3  * 

0.060 

because  of  child  care  problems  (%) 

11.3 

15.3 

-4.0  * 

0.085 

Respondent  missed  work  because  of  child  care  problems  (%) 

2.2 

2.1 

0.1 

0.939 

TransDortation 

Commuting  time  to  current  job,  in  minutes  (%) 

Not  currently  working 

37.5 

38.1 

-0.6 

0.860 

0-15 

32.7 

32.7 

0.0 

0.998 

16-30 

20.7 

18.2 

2.4 

0.385 

31-45 

5.2 

6.3 

-1.2 

0.473 

46  or  more 

3.7 

4.5 

-0.8 

0.569 

Respondent  has  valid  driver's  license  (%) 

76.3 

74.9 

1.4 

0.631 

Respondent  has  access  to  a car  (%) 

68.3 

67.4 

0.9 

0.767 

Method  of  transportation  to  current  jobb  (%) 

By  car 

49.5 

48.6 

0.9 

0.796 

By  bus  or  other  transportation 

4.9 

5.8 

-0.8 

0.611 

Gets  a ride 

8.6 

10.9 

-2.3 

0.266 

Walks 

4.0 

2.3 

1.7 

0.183 

During  past  month,  missed  work  due 

to  problems  with  transportation  (%) 

2.0 

1.5 

0.5 

0.594 

Sample  size  (total  = 892) 

504 

388 

(continued) 

452 


Appendix  Table  D.35  (continued) 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 

NOTES:  The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 

aInfonnal  child  care  is  defined  as  a regular  child  care  arrangement  for  which  neither  the  respondent  nor  any 
other  person  or  institution  paid. 

bRespondents  can  select  more  than  one  method  of  transportation. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.36 

Impacts  on  Child  Care  for  Children  Ages  5 to  8 at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

TvDe  of  child  care  arrangement  in  past  month 

Currently  any  child  care  (%) 

56.8 

51.8 

5.0 

0.383 

Number  of  child  care  arrangements 

0.6 

0.6 

0.1 

0.336 

Any  home-based  care  (%) 

45.0 

45.6 

-0.6 

0.921 

Family  daycare 

6.2 

6.2 

0.1 

0.982 

Other  biological  parent 

4.6 

0.7 

3.8  * 

0.051 

Sibling 

6.6 

3.6 

3.0 

0.231 

Other  relative 

24.9 

31.2 

-6.3 

0.217 

Spouse/partner 

1.1 

0.6 

0.5 

0.669 

Nonrelative  in  child's  home 

4.5 

5.1 

-0.7 

0.786 

Any  center-based  care  (%) 

11.7 

6.3 

5.4 

0.116 

Head  Start 

0.6 

-0.1 

0.6 

0.329 

Preschool,  nursery,  or  daycare  center 

8.4 

2.4 

6.0  ** 

0.029 

After-school  program 

2.7 

4.0 

-1.2 

0.555 

Summer  camp 

0.0 

0.0 

0.0 

0.000 

Extent  of  child  care  in  a tvDical  week 

Number  of  hours  in  child  care 

15 

11 

4 * 

0.059 

0 hours  in  child  care  (%) 

43.7 

48.8 

-5.1 

0.378 

Fewer  than  20  hours  in  child  care  (%) 

30.0 

22.6 

7.4 

0.144 

20  or  more  hours  in  child  care  (%) 

26.3 

28.6 

-2.3 

0.663 

Sample  size  (total  = 338) 

197 

141 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 

NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  1 8 who  were  selected  at  random  by  MDRC 
at  the  time  of  the  interview. 

The  p- value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.37 


Impacts  on  School  Outcomes  and  Behavioral  Problems  for  Children  Ages  5 to  8 

at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

School  outcomes3  (%) 

School  performance 

Above  average 

67.4 

69.4 

-1.9 

0.715 

Below  average 

6.7 

9.2 

-2.5 

0.415 

Ever  in  special  education 

9.8 

14.8 

-5.0 

0.191 

Ever  repeated  a grade 

17.7 

10.2 

7.6  * 

0.062 

Ever  suspended 

7.0 

8.7 

-1.7 

0.574 

Ever  expelled 

2.2 

0.5 

1.7 

0.244 

Behavioral  Droblems3  (%) 

Child  has  behavior  that  is  hard  to  control 

11.6 

14.4 

-2.8 

0.472 

Sample  size  (total  = 338) 

197 

141 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by 
MDRC  at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.38 

Impacts  on  School  Outcomes  for  Children  Ages  13  to  17  at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

School  outcomes3  (%) 

School  performance 
Above  average 

45.6 

43.1 

2.5 

0.756 

Below  average 

19.9 

18.5 

1.4 

0.837 

Ever  in  special  education 

19.1 

17.1 

2.0 

0.740 

Ever  repeated  a grade 

14.8 

12.6 

2.3 

0.698 

Ever  suspended 

31.6 

23.2 

8.4 

0.243 

Ever  expelled 

11.4 

9.7 

1.7 

0.739 

Ever  dropped  out  of  school 

2.5 

6.1 

-3.5 

0.265 

Sample  size  (total  =189) 

98 

91 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42-Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC 
at  the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.39 

Impacts  on  Behavioral  Problems  and  Police  Involvement  Outcomes 
for  Children  Ages  13  to  17  at  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Behavioral  problems3 1%) 

Child  has  behavior  that  is  hard  to  control 

11.9 

11.3 

0.6 

0.914 

Ever  had  a problem  with  alcohol  or  drugs 

6.7 

6.0 

0.7 

0.861 

Police  involvement  outcomes3  (%) 

Ever  in  juvenile  court 

10.3 

9.7 

0.6 

0.906 

Ever  in  trouble  with  the  police 

5.8 

9.2 

-3.4 

0.454 

Sample  size  (total  = 189) 

98 

91 

SOURCE:  MDRC  calculations  from  responses  to  the  ERA  42 -Month  Survey. 


NOTES:  The  sample  comprises  children  between  the  ages  of  5 and  18  who  were  selected  at  random  by  MDRC  at 
the  time  of  the  interview. 

The  p-value  indicates  the  likelihood  that  the  difference  between  the  program  and  control  groups  arose  by 
chance. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
Rounding  may  cause  slight  discrepancies  in  calculating  sums  and  differences. 

The  outcome  measures  are  discussed  in  Chapter  2. 
aAs  reported  by  respondent. 
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Response  Analysis  for  the 
ERA  42-Month  Survey 


The  Employment  Retention  and  Advancement  (ERA)  42 -Month  Survey  is  a source  of 
data  on  the  research  sample  members’  employment,  educational  activities,  household  character- 
istics, housing,  health  and  health  care  coverage,  income,  and  child  outcomes. 

To  be  eligible  for  participation  in  the  42 -month  survey,  individuals  had  to  meet  the 
following  criteria:  membership  in  the  report  sample  for  the  given  site,  age  1 8 or  older,  single 
parent,  and  fluency  in  English  or  Spanish.  The  survey  was  conducted  only  for  three  ERA  tests 
— those  examining  the  Chicago,  Los  Angeles  Reach  for  Success  (RFS),  and  Riverside  Post- 
Assistance  Self-Sufficiency  (PASS)  programs.  The  number  of  42-month  survey  completions 
ranged  from  892  (in  the  Riverside  PASS  test)  to  1,023  (in  the  Chicago  test).  These  sample 
sizes  are  adequate  for  the  purposes  of  detecting  statistically  significant  effects  on  survey 
outcomes. 

The  survey  response  analysis  described  in  this  appendix  evaluates  the  generalizability 
and  unbiasedness  of  the  42 -month  survey  findings  to  each  of  the  three  test’s  research  samples. 
For  each  of  the  three  ERA  tests,  the  analysis  explains  the  process  of  survey  sample  selection, 
reports  the  survey  response  rates,  and  compares  the  background  characteristics  of  respondents 
with  those  of  eligible  nonrespondents  and  the  background  characteristics  of  program  group 
respondents  with  those  of  control  group  respondents.  The  analysis  also  compares  the  adminis- 
trative records  impacts  for  those  in  the  survey-eligible  sample,  the  fielded  sample,  the  respon- 
dent sample,  and  the  report  sample.  Box  D.  1 defines  the  key  samples  used  in  the  analysis.  For 
the  final  set  of  comparisons,  consistent  impacts  are  considered  to  be  the  best  result,  providing 
evidence  that  the  survey  respondent  sample  is  representative  of  other  samples  and  that  the 
survey  results  can  be  generalized  to  the  broader  full  report  sample. 


Background:  Survey  Response  Analysis 

Random  assignment  ensures  that  the  only  systematic  difference  between  the  program 
group  and  the  control  group  is  that  one  group  was  assigned  to  the  program  and  the  other  was 
not.  As  a result,  differences  that  emerge  after  random  assignment  can  be  attributed  to  the 
program  being  studied,  with  some  degree  of  confidence.  This  argument  holds,  however,  only  if 
one  observes  outcomes  for  the  entire  program  group  and  the  entire  control  group  or  outcomes 
for  subgroups  defined  by  baseline  characteristics. 

By  contrast,  surveys  that  rely  on  individuals  volunteering  information  do  not  guarantee 
that  respondents  in  the  program  and  control  groups  will  be  similar  to  each  other.  Because  only  a 
subset  of  the  full  sample  responds  to  a survey,  survey-based  impacts  might  yield  different 
results  than  if  one  has  information  on  the  entire  sample.  For  example,  the  first  survey  respon- 
dents might  be  easier  to  find  and  be  more  willing  to  participate  in  a survey  because  they  are  less 
likely  to  have  moved  since  entering  the  study.  In  addition,  program  group  members  are  some- 
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times  easier  to  find  than  control  group  members  because  the  program  can  provide  more  recent 
contact  information  for  them. 


Box  D.1 

Terms  Used  in  the  Survey  Response  Analysis 

Report  sample.  Sample  members  included  in  the  impact  analysis  for  this  report. 

Survey-eligible  (eligible)  sample.  Research  sample  members  who  met  the  survey  eligibility 
criteria.  For  the  42-month  survey,  survey-eligible  sample  members  were  randomly  selected 
from  all  months  of  random  assignment  in  the  Los  Angeles  RFS  and  Riverside  PASS  tests  and 
from  a majority  of  months  of  random  assignment  in  the  Chicago  test. 

Fielded  sample.  Survey-eligible  sample  members  who  were  randomly  selected  to  be  inter- 
viewed. Because  of  the  expense  of  fielding  a survey,  it  is  common  for  research  projects  such 
as  ERA  to  select  only  a portion  of  the  total  research  sample  for  a survey.  The  fielded  sample 
contains  the  individuals  to  be  located  and  interviewed. 

Respondent  sample.  Fielded  sample  members  who  completed  the  survey.  Attempts  are 
made  to  contact  all  individuals  in  the  fielded  sample,  but  (as  always)  not  everyone  can  be 
located  and  interviewed.  The  proportion  of  the  fielded  sample  that  completes  an  interview 
represents  the  response  rate. 

Nonrespondent  sample.  Fielded  sample  members  who  were  not  interviewed  because  they 
could  not  be  located,  refused  to  be  interviewed,  or  were  unable  to  be  interviewed  for  other 
reasons. 


It  is  common  to  find  some  differences  in  response  rates  or  baseline  characteristics  be- 
tween program  and  control  group  survey  respondents  or  some  differences  in  program  impacts 
between  respondents  and  nonrespondents.  It  is  important  to  determine  how  large  these  differ- 
ences are  and  whether  and  how  they  might  affect  the  results  of  the  analysis. 

Two  important  concepts  to  consider  regarding  the  validity  of  survey  results  are  general- 
izability  and  bias.  If  differences  are  found  between  survey  respondents  and  nonrespondents, 
concerns  emerge  that  the  results  in  the  survey  are  not  applicable,  or  “generalizable,”  to  the  full 
report  sample.  Comparisons  are  also  made  between  program  and  control  group  members  who 
responded  to  the  survey.  If  program  group  respondents  entered  the  evaluation  with  substantially 
different  background  characteristics  than  control  group  respondents,  the  results  may  not 
accurately  represent  the  effects  of  the  program,  and,  therefore,  the  results  of  the  survey  may  be 
“biased.”  This  can  happen,  for  example,  if  program  group  members  are  more  motivated  to 
respond  to  the  survey  because  they  had  a positive  experience  with  the  program.  If  this  were  the 
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case,  different  types  of  program  group  members  would  be  compared  with  control  group 
members,  which  would  undermine  the  key  analytical  underpinning  of  random  assignment:  that 
the  groups  were  identical  at  the  time  they  entered  the  study. 

The  following  section  presents  an  analysis  that  evaluates  the  generalizability  and  unbi- 
asedness of  the  42 -month  survey  findings  for  each  of  the  three  ERA  tests  in  which  that  survey 
was  administered:  Chicago,  Los  Angeles  RFS,  and  Riverside  PASS. 


Chicago 

Survey  Sample  Selection 

The  Chicago  report  sample  includes  1,729  individuals  who  were  randomly  assigned  to 
either  the  program  group  or  the  control  group  between  February  2002  and  March  2003.  (This 
time  frame  includes  almost  all  of  the  random  assignment  period  in  Chicago,  which  extended 
from  February  2002  to  June  2003.) 

Box  D.2  shows  that  1,613  sample  members  were  eligible  for  the  survey  in  Chicago  (93 
percent  of  the  report  sample)  and  that  1,314  were  randomly  selected  to  be  part  of  the  fielded 
sample  (658  program  group  members  and  656  control  group  members). 


Box  D.2 

Key  Analysis  Samples:  Chicago 

Report  sample.  Sample  members  who  were  randomly  assigned  during  the  sample  intake 
period,  which  ranged  from  February  2002  through  March  2003.  (N  = 1,729) 

Survey-eligible  sample.  Research  sample  members  who  met  the  following  eligibility 
criteria:  residence  in  Cook  County,  minimum  age  of  18,  single-parent  status,  and  ability  to 
speak  English  or  Spanish.  (N  = 1,613) 

Fielded  sample.  Eligible  sample  members  who  were  randomly  selected  from  the  survey- 
eligible  sample  to  be  interviewed.  (N  = 1,314) 

Respondent  sample.  Sample  members  in  the  fielded  sample  who  completed  the  ERA  42- 
Month  Survey.  (N  = 1,023) 

Nonrespondent  sample.  Sample  members  in  the  fielded  sample  who  were  not  interviewed 
because  they  were  not  located,  refused  to  be  interviewed,  were  located  after  the  fielded 
period  expired,  were  incarcerated,  or  were  unable  to  be  interviewed  for  other  reasons.  (N  = 
291) 
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Survey  Response  Rates 

Appendix  Table  D.40  shows  that,  overall,  1,023  sample  members  (78  percent  of  the 
fielded  sample)  responded  to  the  survey.  Among  the  291  fielded  sample  members  who  were  not 
interviewed,  258  could  not  be  located;  22  refused  to  respond;  9 were  located  after  the  fielded 
period  had  expired;  1 was  incarcerated;  and  1 did  not  respond  for  other  reasons.  Response  rates 
for  the  two  research  groups  are  similar:  79  percent  for  the  ERA  group  and  77  percent  for  the 
control  group. 

Comparison  of  the  Background  Characteristics  of  Respondents  and 

Nonrespondents  in  the  Fielded  Sample 

As  described  above,  it  is  important  to  examine  whether  any  differences  exist  between 
the  background  characteristics  of  survey  respondents  and  nonrespondents  in  order  to  determine 
whether  the  survey  results  are  adequately  generalizable  to  the  full  report  sample.  To  determine 
whether  systematic  differences  exist  between  survey  respondents  and  nonrespondents,  an 
indicator  of  survey  response  was  created,  and  multivariate  regression  analysis  was  used  to 
identify  whether  baseline  characteristics  — related  to  age,  gender,  race/ethnicity,  number  and 
age  of  children,  educational  attainment,  prior  employment  and  earnings,  receipt  of  public 
benefits,  and  research  group  — are  significantly  associated  with  survey  response. 

Appendix  Table  D.41  shows  the  logistic  regression  coefficients  and  p-values  for  the 
probability  of  being  a survey  respondent.  The  parameter  estimates  in  the  first  column  provide  a 
measure  of  the  relationship  between  each  baseline  variable  and  survey  response,  and  the 
asterisks  and  p-values  show  the  statistical  significance  of  these  relationships.  Statistically 
significant  effects  were  found  for  employment  in  the  quarter  prior  to  random  assignment,  total 
grant  amount  from  the  Temporary  Assistance  for  Needy  Families  (TANF)  program  in  the 
month  prior  to  random  assignment,  and  length  of  TANF  receipt  in  the  prior  year  — indicating 
that  survey  respondents,  compared  with  nonrespondents,  had  slightly  higher  rates  of  employ- 
ment and  TANF  receipt  prior  to  random  assigmnent. 

Although  statistically  significant  associations  between  baseline  characteristics  and  sur- 
vey response  are  cause  for  concern,  the  coeffecients  for  these  associations  for  the  Chicago 
survey  are  small,  and  the  overall  model  shows  minimal  differentiation  in  baseline  characteristics 
between  survey  respondents  and  nonrespondents.  Consequently,  baseline  differences  between 
survey  respondents  and  nonrespondents  were  deemed  to  be  unlikely  to  negatively  affect  the 
generalizability  of  the  survey  results. 
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Comparison  of  the  Background  Characteristics  of  Program  and  Control 

Group  Members  in  the  Survey  Respondent  Sample 

Because  ERA  was  a random  assignment  study,  program  and  control  group  members  in 
the  report  sample  had  similar  characteristics.  As  discussed  above,  however,  this  does  not  assure 
that,  among  survey  respondents,  program  and  control  group  members  will  share  similar  charac- 
teristics. Using  difference  of  means  t-tests  and  chi-square  analysis,  Appendix  Table  D.42 
compares  the  means  of  baseline  variables  related  to  respondents’  children,  race/ethnicity, 
gender,  age,  English  fluency,  prior  employment,  prior  earnings,  and  prior  receipt  of  TANF  for 
program  and  control  group  members  who  responded  to  the  survey. 

In  general,  differences  between  program  and  control  group  respondents  are  small  and  not 
statistically  significant.  The  exception  is  age  of  youngest  child,  for  which  the  mean  for  the 
program  group  is  5.7  years  and  the  mean  for  the  control  group  is  6.3  years.  This  moderate  differ- 
ence was  deemed  to  be  unlikely  to  greatly  affect  the  unbiasedness  of  the  survey  results  because  of 
the  small  size  of  the  difference  and  because  it  is  unclear  how  or  whether  the  age  of  youngest  child 
would  affect  sample  members’  interactions  with  the  Chicago  ERA  program  under  study. 

Finally,  a multivariate  analysis  was  conducted  — one  similar  to  the  analysis  discussed 
in  the  preceding  section,  except  that  the  dependent  variable  of  this  model  is  research  group 
status  and  the  model  was  run  among  survey  respondents  only.  Overall,  this  model  is  not 
statistically  significant,  providing  further  assurance  that  the  survey  results  are  unbiased. 

Comparison  of  the  Economic  Impacts  for  the  Survey-Eligible,  Fielded, 

Respondent,  and  Report  Samples 

The  final  aspect  of  the  analysis  used  data  from  administrative  records  to  compare  im- 
pacts across  the  eligible,  fielded,  respondent,  and  report  samples. 

Appendix  Table  D.43  shows  research  group  means  and  impacts  on  employment,  earn- 
ings, and  TANF  receipt  for  the  four  samples,  using  several  key  measures  from  the  report.  The 
table  shows  that  most  impacts  are  similar  in  direction  and  magnitude  across  these  samples. 
There  are  minor  exceptions;  for  example,  impacts  on  average  quarterly  employment  and 
cumulative  earnings  are  generally  largest  for  the  report  and  the  survey-eligible  samples  and 
smallest  for  the  respondent  sample. 

Conclusions:  Chicago 

This  analysis  shows  that  survey  respondents  and  nonrespondents  have  similar  baseline 
characteristics,  as  do  program  group  survey  respondents  and  control  group  survey  respondents. 
Although  some  differences  in  the  characteristics  of  program  and  control  group  respondents 
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exist,  the  differences  are  not  large  enough  to  cause  concern  about  the  validity  of  survey  results. 
Accordingly,  there  are  no  major  hesitations  about  generalizing  the  survey  results  to  the  report 
sample  or  about  whether  the  survey  impacts  are  unbiased.  Further  confidence  derives  from  the 
observation  that  four-year  impacts  are  generally  similar  across  the  four  analysis  samples. 
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The  Employment  Retention  and  Advancement  Project 

Appendix  Table  D.40 

Response  Rates  for  the  ERA  42-Month  Survey: 
Chicago 


ERA 

Group 

Control 

Group 

Total 

Number  from  survey-eligible  sample 

800 

813 

1,613 

Number  from  fielded  sample 

658 

656 

1,314 

Number  who  responded  to  the  survey 

521 

502 

1,023 

Response  rate  (%) 

79.2 

76.5 

77.9 

SOURCE:  MDRC  calculations  based  on  sampling  and  survey  administration  results  for  the  ERA  42- 
Month  Survey. 
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The  Employment  Retention  and  Advancement  Project 

Appendix  Table  D.41 

Estimated  Regression  Coefficients  for  the  Probability  of  Being  a 
Respondent  to  the  ERA  42-Month  Survey: 

Chicago 


Survey  Sample 
Parameter 

Estimate  P-Value 

ERA  group 

0.177 

0.193 

Female 

0.199 

0.819 

Age 

0.035 

0.181 

3 1 -40  years  old 

-0.121 

0.632 

41  years  old  or  older 

-0.445 

0.361 

Black,  non-Hispanic 

0.397 

0.173 

Hispanic 

-0.086 

0.821 

No  high  school  diploma  or  GED  certificate 

0.081 

0.573 

Speak  limited  English 

0.551 

0.341 

Age  of  youngest  child 

-0.002 

0.916 

Number  of  children 

-0.015 

0.825 

Employed  in  the  prior  year 

-0.366 

0.278 

Employed  in  the  prior  quarter 

0.579  * 

0.079 

Number  of  quarters  employed  in  the  prior  year 

0.081 

0.564 

Ever  employed  in  past  3 years 

-0.211 

0.143 

Earnings  in  the  prior  year 

< .001 

0.213 

Total  TANF  grant 

0.001  ** 

0.030 

Received  TANF  in  the  prior  year 

0.182  ** 

0.048 

Relative  month  of  random  assignment 

0.003 

0.851 

R-square  (0.0251) 

Wald-statistic  (32.671 1) 

P-value  of  Wald-statistic  (0.0262) 

Sample  size 

1,314 

SOURCE:  MDRC  calculations  from  ERA  baseline  forms  and  administrative  data. 

NOTE:  Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.42 

Background  Characteristics  of  42-Month  Survey  Respondents: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Female  (%) 

99.6 

99.4 

Race/ethnicity  (%) 
Black,  non-Hispanic 

89.1 

88.7 

Hispanic 

6.7 

7.0 

Age 

3 1 -40  years  old  (%) 

46.5 

48.2 

40  years  old  or  over  (%) 

16.7 

16.9 

Average  age  (years) 

33 

34 

No  high  school  diploma  or  GED  certificate  (%) 

54.9 

56.2 

Speak  limited  English  (%) 

1.2 

2.4 

Ever  employed  (%) 

59.3 

55.8 

Employed  in  prior  year  (%) 

66.4 

66.5 

Employed  in  prior  quarter  (%) 

59.9 

59.2 

Number  of  quarters  employed  in  prior  year 

2.3 

2.3 

Earnings  in  prior  year  ($) 

5,416 

5,461 

Number  of  children  (%) 

3.3 

3.2 

Age  of  youngest  child  (years) 

5.7 

6.3  ** 

Total  TANF  grant  ($) 

170 

177 

Received  TANF  in  prior  year  (%) 

11.9 

11.9 

Sample  size  (total  = 1,023) 

521 

502 

SOURCE:  MDRC  calculations  from  baseline  data  and  administrative  records. 

NOTES:  Chi-square  (categorical)  tests  and  two-tailed  (continuous)  t-tests  were  used  to  assess  the 
difference  in  characteristics  between  the  ERA  and  control  groups. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.43 


Comparision  of  Impacts  for  the  Report,  Fielded,  Eligible, 
and  Respondent  Samples  with  the  42-Month  Survey: 

Chicago 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P -Value 

Ouarters  2-17 

Ever  employed  (%) 

Report  sample 

80.5 

79.3 

1.2 

0.447 

Eligible  sample 

80.3 

79.1 

1.2 

0.473 

Fielded  sample 

79.7 

78.1 

1.6 

0.416 

Respondent  sample 

80.8 

79.7 

1.2 

0.586 

Average  quarterly  employment  (%) 

Report  sample 

55.0 

52.3 

2.7 

0.055  * 

Eligible  sample 

55.1 

52.8 

2.3 

0.119 

Fielded  sample 

54.1 

52.8 

1.3 

0.422 

Respondent  sample 

56.1 

54.9 

1.2 

0.521 

Employed  4 consecutive  quarters  (%) 

Report  sample 

25.8 

22.9 

2.9 

0.111 

Eligible  sample 

26.6 

23.4 

3.2 

0.097  * 

Fielded  sample 

25.5 

23.6 

1.9 

0.365 

Respondent  sample 

26.0 

24.6 

1.5 

0.543 

Number  of  quarters  employed 

Report  sample 

8.8 

8.4 

0.4 

0.056  * 

Eligible  sample 

8.8 

8.4 

0.4 

0.120 

Fielded  sample 

8.7 

8.4 

0.2 

0.422 

Respondent  sample 

9.0 

8.8 

0.2 

0.521 

Average  annual  earnings  ($) 

Report  sample 

6,979 

6,479 

500 

0.061  * 

Eligible  sample 

7,007 

6,562 

445 

0.109 

Fielded  sample 

6,960 

6,661 

299 

0.333 

Respondent  sample 

7,108 

6,815 

292 

0.398 

Ouarters  2-13 

Ever  received  TANF  (%) 

Report  sample 

86.1 

89.2 

-3.1 

0.052  * 

Eligible  sample 

86.7 

89.3 

-2.6 

0.107 

Fielded  sample 

86.0 

89.5 

-3.5 

0.051  * 

Respondent  sample 

87.2 

90.2 

-3.0 

0.126 

Average  annual  TANF  received  ($) 

Report  sample 

4,111 

3,958 

153 

0.050  ** 

Eligible  sample 

4,081 

3,930 

151 

0.060  * 

Fielded  sample 

4,115 

3,891 

225 

0.013  ** 

Respondent  sample 

4,226 

4,109 

116 

0.229 

(continued) 
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Appendix  Table  D.43  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Average  annual  income  ($) 

Report  sample 

11,480 

11,106 

374 

0.132 

Eligible  sample 

11,474 

11,170 

304 

0.239 

Fielded  sample 

11,444 

11,228 

216 

0.456 

Respondent  sample 

11,709 

11,618 

91 

0.769 

SOURCE:  MDRC  calculations  from  the  state  administrative  records. 

NOTES:  The  research  sample  includes  1,728  sample  members;  ERA  group:  854;  control  group:  874.  The  eligible 
sample  includes  1,613  sample  members;  ERA  group:  800;  control  group:  813.  The  fielded  sample  includes  1,314 
sample  members;  ERA  group:  658;  control  group:  656.  The  respondent  sample  includes  1,023  sample  members; 
ERA  group:  521 ; control  group:  502. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
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Los  Angeles  RFS 


Survey  Sample  Selection 

The  report  sample  for  the  Los  Angeles  Reach  for  Success  (RFS)  test  includes  5,702 
sample  members  who  were  randomly  assigned  to  the  ERA  program  group  or  to  the  control 
group  between  July  2002  and  July  2004  (the  full  random  assignment  period).  Box  D.3  shows 
that  73  percent  of  the  report  sample  members  met  the  survey  eligibility  criteria  and  that,  of  those 
eligible,  1,344  were  randomly  selected  for  the  fielded  sample,  split  evenly  between  the  program 
group  (N  = 674)  and  the  control  group  (N  = 670). 

Box  D.3 

Key  Analysis  Samples:  Los  Angeles  RFS 

Report  sample.  Sample  members  who  were  randomly  assigned  during  the  sample  intake 
period,  which  ranged  from  July  2002  through  July  2004.  (N  = 5,702) 

Survey-eligible  sample.  Research  sample  members  who  met  the  following  criteria  for 
inclusion:  minimum  age  of  18,  single-parent  status,  and  ability  to  speak  English  or  Spanish. 

(N  = 4,142) 

Fielded  sample.  Eligible  sample  members  who  were  randomly  selected  from  the  survey- 
eligible  sample  to  be  interviewed.  (N  = 1 ,344) 

Respondent  sample.  Sample  members  in  the  fielded  sample  who  completed  the  ERA  42- 
Month  Survey.  (N  = 982) 

Nonrespondent  sample.  Sample  members  in  the  fielded  sample  who  were  not  interviewed 
because  they  were  not  located,  refused  to  be  interviewed,  were  located  after  the  fielded 
period  expired,  were  incarcerated,  or  were  unable  to  be  interviewed  for  other  reasons.  (N  = 

362) 


Survey  Response  Rates 

Appendix  Table  D.44  shows  that  982  sample  members,  or  73  percent  of  the  fielded 
sample,  were  interviewed  for  the  survey.  Among  the  360  fielded  sample  members  who  were  not 
interviewed,  275  could  not  be  located;  62  refused  to  respond;  21  were  located  after  the  fielded 
period  had  expired;  and  2 were  incarcerated;  in  addition,  2 sample  members  completed  only 
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Comparison  of  the  Background  Characteristics  of  Respondents  and 

Nonrespondents  in  the  Fielded  Sample 

Appendix  Table  D.45  shows  the  parameter  estimates  and  p-values  for  the  probability  of 
responding  to  the  survey.  There  are  some  differences  between  survey  respondents  and  nonre- 
spondents. For  example,  survey  respondents  were  more  likely  to  be  black,  Hispanic,  and  long- 
term recipients  of  Aid  to  Families  with  Dependent  Children  (AFDC),  and  they  were  less  likely 
to  be  residing  in  Regions  1 and  5 of  the  Greater  Avenues  for  Independence  (GAIN)  program  — 
California’s  mandatory  welfare-to-work  program.  However,  these  differences  are  not  large, 
which  is  highlighted  by  the  fact  that  the  regression  model  described  above  is  not  statistically 
significant  overall.  (In  other  words,  the  overall  model  is  unable  to  distinguish  statistically 
between  survey  respondents  and  nonrespondents;  the  Wald-statistic  is  not  statistically  signifi- 
cant.) Consequently,  the  differences  were  deemed  to  be  unlikely  to  affect  the  generalizability  of 
the  survey  results. 

Comparison  of  the  Background  Characteristics  of  Program  and  Control 

Group  Members  in  the  Survey  Respondent  Sample 

Appendix  Table  D.46  compares  the  program  and  control  group  respondents’  means  for 
baseline  variables  related  to  race/ethnicity,  gender,  age,  number  of  children,  English  fluency, 

prior  employment,  prior  earnings,  and  prior  receipt  of  public  assistance  — to  check 
whether  the  background  characteristics  of  program  and  control  group  respondents  are  similar. 
There  are  two  statistically  significant  differences  between  program  and  control  group  respon- 
dents. Survey  respondents  in  the  program  group  received  TANF  for  fewer  months  prior  to 
random  assignment  and  were  less  likely  to  be  residing  in  GAIN  Region  5 than  respondents  in 
the  control  group.  These  differences  do  not  relate  to  background  characteristics  that  are  ex- 
pected to  moderate  program  impacts  and,  therefore,  were  deemed  to  be  unlikely  to  greatly 
influence  the  validity  of  the  survey  impact  results. 

Finally,  a multivariate  analysis  was  conducted  to  determine  whether  several  background 
characteristics  could  predict  research  group  status  among  the  survey  respondent  sample. 
Overall,  this  model  is  not  statistically  significant,  providing  further  assurance  that  the  survey 
results  are  unbiased. 

Comparison  of  the  Economic  Impacts  for  the  Survey-Eligible,  Fielded, 

Respondent,  and  Report  Samples 

Appendix  Table  D.47  shows  research  group  means  and  impacts  on  employment,  earn- 
ings, TANF,  and  food  stamp  outcomes  for  the  four  analysis  samples.  With  small  exceptions,  the 
results  are  generally  consistent  across  the  four  analysis  samples,  providing  additional  assurance 
about  the  generalizability  of  the  survey  results. 
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Conclusions:  Los  Angeles  RFS 

These  findings  suggest  that  the  survey  results  can  be  generalized  from  the  survey  re- 
spondent sample  to  the  broader  report  sample.  The  fact  that  survey  respondents  and  survey- 
eligible  nonrespondents  as  well  as  program  group  and  control  group  respondents  share  similar 
baseline  characteristics  suggests  that  little,  if  any,  response  bias  exists  and  that  the  survey 
respondent  sample  is  representative  of  other  analysis  samples.  This  analysis  found  no  grounds 
for  concern  about  the  generalizability  or  unbiasedness  of  the  survey  results  in  Los  Angeles  RFS. 
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The  Employment  Retention  and  Advancement  Project 

Appendix  Table  D. 44 

Response  Rates  for  the  ERA  42-Month  Survey: 
Los  Angeles  RFS 


ERA 

Group 

Control 

Group 

Total 

Number  from  survey-eligible  sample 

2,079 

2,063 

4,142 

Number  from  fielded  sample 

674 

670 

1,344 

Number  who  responded  to  the  survey 

498 

484 

982 

Response  rate  (%) 

73.9 

72.2 

73.1 

SOURCE:  MDRC  calculations  based  on  sampling  and  survey  administration  for  the  ERA  42 -Month 
Survey. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.45 

Estimated  Regression  Coefficients  for  the  Probability  of  Being  a 
Respondent  to  the  ERA  42-Month  Survey: 

Los  Angeles  RFS 


Survey  Sample 
Parameter 

Estimate  P-Value 

ERA  group 

0.068 

0.591 

Female 

0.020 

0.941 

Age 

0.015 

0.150 

No  high  school  diploma  or  GED  certificate 

0.060 

0.655 

Black,  non-Hispanic 

0.722  *** 

0.001 

Hispanic 

0.418  * 

0.057 

Other  race/ethnicity 

0.560 

0.283 

Youngest  child  age  3-5 

0.082 

0.623 

Younges  child  age  6-12 

-0.105 

0.623 

Youngest  child  age  13-18 

-0.171 

0.584 

Number  of  children 

-0.007 

0.388 

Employed  in  the  prior  year 

0.009 

0.964 

Employed  in  the  prior  quarter 

-0.204 

0.329 

Received  food  stamps  in  the  prior  year 

-0.501 

0.218 

Received  TANF  in  the  prior  year 

-0.009 

0.771 

Long-term  TANF  receipt 

0.308  ** 

0.026 

GAIN  Region  1 

-0.625  *** 

0.001 

GAIN  Region  5 

-0.375  ** 

0.028 

Relative  month  of  random  assignment 

-0.005 

0.641 

R-square  (0.0257) 

Wald-statistic  (33.7751) 

P-value  of  Wald-statistic  (0.2899) 

Sample  size 

1,347 

SOURCE:  MDRC  calculations  from  ERA  baseline  forms  and  administrative  data. 


NOTE:  Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.46 


Background  Characteristics  of  42-Month  Survey  Respondents: 

Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Female  (%) 

93.2 

94.6 

Race/ethnicity  (%) 

47.8 

46.8 

Hispanic 

42.0 

43.9 

Black 

2.2 

1.2 

Other 

Average  age  (years) 

31 

31 

No  high  school  diploma  or  GED  certificate  (%) 

51.2 

49.2 

Employed  during  the  quarter  prior  to  random  assignment  (%) 

48.6 

44.0 

Employed  during  the  year  prior  to  random  assignment  (%) 

68.3 

65.1 

Number  of  quarters  employed  in  the  prior  3 years  (%) 

5.4 

5.1 

Earnings  in  the  3 years  prior  to  random  assignment  ($) 

14,009 

13,082 

Number  of  children  (%) 

2.0 

2.0 

Age  of  youngest  child  (%) 

Under  3 years 

39.0 

39.7 

3-5  years 

42.2 

39.7 

6-12  years 

55.8 

55.2 

13-18  years 

32.3 

28.9 

TANF  receipt  history  (%) 

Received  TANF  in  year  prior  to  random  assignment 

96.0 

96.1 

Number  of  months  received  TANF  in  prior  year 

8.7 

9.2  * 

Received  TANF  2 years  or  more  prior  to  random  assignment 

50.4 

53.5 

Received  food  stamps  in  prior  year  (%) 

90.2 

89.9 

GAIN  Region  1 

27.9 

25.8 

GAIN  Region  5 

41.6 

47.3  * 

Sample  size  (total  = 982) 

498 

484 

SOURCE:  MDRC  calculations  from  baseline  data  and  administrative  records. 

NOTES:  Chi-square  (categorical)  tests  and  two-tailed  (continuous)  t-tests  were  used  to  assess  the  difference  in 
characteristics  between  the  ERA  and  control  groups. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
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SOURCE:  MDRC  calculations  from  baseline  data  and  administrative  records. 


NOTES:  Chi-square  (categorical)  tests  and  two-tailed  (continuous)  t-tests  were  used  to  assess  the  difference  in 
characteristics  between  the  ERA  and  control  groups. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.47 

Comparision  of  Impacts  for  the  Report,  Fielded,  Eligible, 
and  Respondent  Samples  with  the  42-Month  Survey: 


Los  Angeles  RFS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Ouarters  2-13 

Ever  employed  (%) 

Report  sample 

92.3 

91.5 

0.7 

0.285 

Eligible  sample 

91.5 

91.9 

-0.4 

0.657 

Fielded  sample 

92.3 

93.2 

-0.9 

0.531 

Respondent  sample 

92.4 

93.8 

-1.3 

0.400 

Average  quarterly  employment  (%) 

Report  sample 

63.6 

64.2 

-0.6 

0.495 

Eligible  sample 

64.1 

65.0 

-0.9 

0.368 

Fielded  sample 

65.1 

65.0 

0.1 

0.972 

Respondent  sample 

66.4 

66.5 

-0.2 

0.929 

Employed  4 consecutive  quarters  (%) 

Report  sample 

29.1 

28.2 

0.9 

0.423 

Eligible  sample 

29.5 

29.4 

0.1 

0.923 

Fielded  sample 

31.6 

28.5 

3.1 

0.196 

Respondent  sample 

32.8 

29.8 

3.0 

0.287 

Number  of  quarters  employed 

Report  sample 

7.6 

7.7 

-0.1 

0.497 

Eligible  sample 

7.7 

7.8 

-0.1 

0.370 

Fielded  sample 

7.8 

7.8 

0.0 

0.967 

Respondent  sample 

8.0 

8.0 

0.0 

0.935 

Average  annual  earnings  (S) 

Report  sample 

9,841 

9,830 

11 

0.961 

Eligible  sample 

10,061 

10,168 

-107 

0.674 

Fielded  sample 

10,380 

9,779 

600 

0.180 

Respondent  sample 

10,518 

10,036 

482 

0.361 

Average  annual  TANF  received  ($) 

Report  sample 

95.1 

94.0 

1.1 

0.058  * 

Eligible  sample 

95.7 

94.0 

1.7 

0.013  ** 

Fielded  sample 

95.1 

94.7 

0.5 

0.711 

Respondent  sample 

95.0 

95.3 

-0.3 

0.825 

Amount  of  food  stamps  received  (S) 

Report  sample 

2,056 

2,011 

46 

0.224 

Eligible  sample 

2,080 

1,999 

80 

0.070  * 

Fielded  sample 

2,025 

2,086 

-61 

0.440 

Respondent  sample 

2,115 

2,176 

-62 

0.511 

(continued) 
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Appendix  Table  D.47  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Average  annual  income  ($) 

Report  sample 

15,210 

15,056 

155 

0.417 

Eligible  sample 

15,417 

15,353 

64 

0.779 

Fielded  sample 

15,574 

15,177 

397 

0.325 

Respondent  sample 

15,962 

15,723 

239 

0.608 

SOURCE:  MDRC  calculations  from  the  state  administrative  records. 

NOTES:The  research  sample  includes  5,700  sample  members;  ERA  group:  2,857;  control  group:  2,843.  The 
eligible  sample  includes  4,145  sample  members;  ERA  group:  2,081;  control  group:  2,064.  The  fielded  sample 
includes  1,347  sample  members;  ERA  group:  676;  control  group:  671.  The  respondent  sample  includes  982 
sample  members;  ERA  group:  498;  control  group:  484. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
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Riverside  PASS 


Survey  Sample  Selection 

The  report  sample  for  the  Riverside  PASS  test  includes  2,770  individuals  who  were 
randomly  assigned  to  either  a program  group  (N  = 1,627)  or  a control  group  (N  = 1,143) 
between  July  2002  and  June  2003  (the  full  random  assignment  period).  Box  D.4  shows  that  the 
survey-eligible  sample  consists  of  1,957  single  parents.  Of  those  eligible,  1,310  were  randomly 
selected  to  be  in  the  fielded  sample  (67  percent  of  the  survey-eligible  sample).  More  program 
group  members  were  selected  than  control  group  members,  reflecting  the  uneven  distribution  of 
program  and  control  group  members  in  the  overall  report  sample. 


Box  D.4 

Key  Analysis  Samples:  Riverside  PASS 

Report  sample.  Sample  members  who  were  randomly  assigned  during  the  sample  intake 
period,  which  ranged  from  July  2002  to  June  2003.  (N  = 2,770) 

Survey-eligible  sample.  Research  sample  members  who  met  the  following  criteria  for 
inclusion:  minimum  age  of  18,  single-parent  status,  and  fluency  in  English  or  Spanish. 
(N=  1,957) 

Fielded  sample.  Eligible  sample  members  who  were  randomly  selected  from  the  survey- 
eligible  sample  to  be  interviewed.  (N  = 1,310) 

Respondent  sample.  Sample  members  in  the  fielded  sample  who  completed  the  ERA  42- 
Month  Survey.  (N  = 892) 

Nonrespondent  sample.  Sample  members  in  the  fielded  sample  who  were  not  interviewed 
because  they  could  not  be  located,  refused  to  be  interviewed,  were  located  after  the  fielded 
period  expired,  were  incarcerated,  were  deceased,  or  were  unable  to  be  interviewed  for 
other  reasons.  (N  = 4 1 8) 
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Survey  Response  Rates 

Appendix  Table  D.48  shows  that  approximately  68  percent  of  the  fielded  sample  mem- 
bers (N  = 892)  responded  to  the  survey.  Among  the  417  sample  members  who  were  not 
interviewed,  339  could  not  be  located;  48  refused  to  be  interviewed;  20  were  located  after  the 
fielded  period  expired;  3 were  institutionalized;  and  7 were  deceased;  in  addition,  1 sample 
member  completed  only  a partial  interview.  Response  rates  differ  somewhat  between  research 
groups:  66  percent  of  program  group  members  and  70  percent  of  control  group  members 
responded  to  the  survey. 

Comparison  of  the  Background  Characteristics  of  Respondents  and 

Nonrespondents  in  the  Fielded  Sample 

Appendix  Table  D.49  shows  the  parameter  estimates  and  p-values  for  the  probability  of 
responding  to  the  42 -month  survey.  Overall,  the  regression  model  is  statistically  significant 
(which  implies  that  survey  respondents  differed  in  some  respects  from  nonrespondents). 
Specifically,  respondents  were  less  likely  than  nonrespondents  to  be  Hispanic  and  to  be  served 
at  the  Center  for  Employment  Training  site,  but  they  were  employed  for  more  quarters  in  the 
prior  three  years.  Because  of  these  differences,  the  comparison  of  impacts  across  the  four 
analysis  samples  is  particularly  important  (discussed  below). 

Comparison  of  the  Background  Characteristics  of  Program  and  Control 

Group  Members  in  the  Survey  Respondent  Sample 

Appendix  Table  D.50  compares  several  baseline  characteristics  of  program  and  control 
group  respondents.  In  general,  baseline  variables  are  similar  across  the  two  research  groups. 
One  exception  is  that  program  group  members  are  somewhat  less  likely  to  have  received  food 
stamps  in  the  year  prior  to  random  assigmnent:  84.7  percent  of  program  group  members 
received  food  stamps,  compared  with  89.4  percent  of  control  group  members.  Since  this 
difference  is  not  large  and  is  not  directly  related  to  employment  retention  and  advancement 
measures,  it  is  not  of  major  concern. 

Finally,  a multivariate  analysis  found  that,  within  the  respondent  sample,  a series  of 
baseline  characteristics  is  not  associated  with  research  group  status,  providing  further  assurance 
that  the  survey  results  are  unbiased. 

Comparison  of  the  Economic  Impacts  for  the  Survey-Eligible,  Fielded, 

Respondent,  and  Report  Samples 

Appendix  Table  D.51  shows  research  group  means  and  impacts  for  the  survey-eligible, 
fielded,  respondent,  and  report  samples.  In  general,  impacts  are  consistent  in  magnitude  and 
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direction,  suggesting  that  the  survey  results  can  be  generalized  beyond  the  survey  respondent 
sample.  While  not  all  samples  show  statistically  significant  differences  for  all  the  measures, 
these  impacts  are  broadly  similar  across  the  four  analysis  samples.  Public  assistance  and  income 
impacts  are  largely  consistent  as  well.  For  example,  the  Riverside  PASS  program  reduced 
TANF  receipt  and  food  stamp  benefit  amounts  for  all  four  samples.  Impacts  on  earnings  and 
total  income  are  consistent  in  direction  and  magnitude  across  the  four  samples,  though  impacts 
for  the  respondent  sample  fail  to  achieve  statistical  significance  for  these  particular  outcomes. 

Conclusion:  Riverside  PASS 

In  sum,  these  findings  show  that  the  survey  findings  can  be  validly  generalized  to  the 
report  sample  in  Riverside  PASS  and  that  the  survey  impact  estimates  are  unbiased.  There  are 
few  differences  in  the  baseline  characteristics  of  survey  respondents  and  nonrespondents  or  of 
program  group  and  control  group  respondents.  In  addition,  impacts  are  largely  consistent  across 
outcomes  and  analysis  samples. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.48 

Response  Rates  for  the  ERA  42-Month  Survey: 
Riverside  PASS 


ERA 

Group 

Control 

Group 

Total 

Number  from  survey-eligible  sample 

1,129 

828 

1,957 

Number  from  fielded  sample 

757 

553 

1,310 

Number  who  responded  to  the  survey 

504 

388 

892 

Response  rate  (%) 

66.1 

70 

68.1 

SOURCE:  MDRC  calculations  based  on  sampling  and  survey  administration  results  for  the  ERA  42- 
Month  Survey. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.49 

Estimated  Regression  Coefficients  for  the  Probability  of  Being  a 
Respondent  to  the  ERA  42-Month  Survey: 

Riverside  PASS 


Survey  Sample 
Parameter 

Estimate  P-Value 

ERA  group 

-0.033 

0.800 

Female 

0.075 

0.723 

Age 

-0.002 

0.822 

Black,  non-Hispanic 

0.107 

0.559 

White 

-0.226 

0.121 

Hispanic 

-0.678  * 

0.051 

No  high  school  diploma  or  GED  certificate 

-0.075 

0.587 

Age  of  the  youngest  child 

-0.023 

0.217 

Number  of  children 

-0.100 

0.141 

Employed  in  the  prior  year 

-0.443 

0.101 

Employed  in  the  prior  quarter 

0.195 

0.358 

Number  of  quarters  employed  in  prior  3 years 

0.072  ** 

0.013 

Earnings  in  the  prior  3 years 

<.001 

0.718 

Received  food  stamps  in  the  prior  year 

<.001 

0.219 

Center  for  Employment  Training 

-0.452  ** 

0.013 

Valley  Restart 

-0.068 

0.697 

Volunteer  Center 

0.214 

0.260 

Relative  month  of  random  assignment 

0.012 

0.612 

R-square  (0.0329) 

Wald-statistic  (42.2073) 

P-value  of  Wald-statistic  (0.001) 

Sample  size 

1,310 

SOURCE:  MDRC  calculations  from  ERA  baseline  forms  and  administrative  data. 

NOTE:  Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10 
percent. 


488 


The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.50 

Background  Characteristics  of  42-Month  Survey  Respondents: 

Riverside  PASS 


ERA 

Control 

Outcome 

Group 

Group 

Female  (%) 

92.1 

89.4 

Race/ethnicity  (%) 

Black,  lioii-Hispaiiic 

17.3 

20.6 

White,  noii-Hispaiiic 

31.4 

28.9 

FUspanic 

48.0 

49.2 

Other 

3.0 

1.3  * 

Age  (%) 

20  or  yoiuiger 

9.7 

9.8 

21-30 

43.5 

42.5 

31-40 

32.9 

30.7 

41  or  older 

13.9 

17.0 

Average  age  (years) 

30.7 

31.1 

High  school  diploma  (%) 

61.1 

56.4 

Employed  during  the  quarter  prior  to  random  assignment  (%) 

79.2 

78.4 

Employed  during  the  year  prior  to  random  assignment  (%) 

87.1 

86.3 

Number  of  quarters  employed  in  the  prior  3 years  (%) 

7.1 

7.1 

Earnings  in  the  3 years  prior  to  random  assignment  ($) 

17,085 

17,473 

Number  of  children  (%) 

0 

0.8 

0.3 

1 

38.7 

39.4 

2 

33.1 

29.6 

More  than  3 

27.4 

30.7 

Average  number  of  children 

2.1 

2.1 

Age  of  youngest  child  (%) 

Under  3 years 

38.7 

36.9 

3-5  years 

24.0 

22.2 

6 years  and  older 

36.5 

40.7 

TANF  receipt  history  (%) 

Never 

3.6 

3.6 

Less  than  3 months 

33.1 

34.0 

3 months  or  more  and  less  than  2 years 

11.1 

10.3 

2 years  or  more  and  less  than  5 years 

20.4 

22.7 

5 years  or  more  and  less  than  10  years 

11.7 

11.6 

10  years  or  more 

7.3 

6.7 

Received  food  stamps  in  prior  year  (%) 

84.7 

89.4  ** 

Sample  size  (total  = (892) 

504 

388 

(continued) 
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Appendix  Table  D.50  (continued) 


SOURCE:  MDRC  calculations  from  baseline  data  and  administrative  records. 

NOTES:  Chi-square  (categorical)  tests  and  two-tailed  (continuous)  t-tests  were  used  to  assess  the  difference  in 
characteristics  between  the  ERA  and  control  groups. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
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The  Employment  Retention  and  Advancement  Project 
Appendix  Table  D.51 

Comparision  of  Impacts  for  the  Report,  Fielded,  Eligible, 
and  Respondent  Samples  with  the  42-Month  Survey: 

Riverside  PASS 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Ouarters  2-17 

Ever  employed  (%) 

Report  sample 

90.5 

88.8 

1.7 

0.131 

Eligible  sample 

91.4 

89.3 

2.0 

0.121 

Fielded  sample 

91.8 

88.9 

2.9 

0.071  * 

Respondent  sample 

95.9 

90.2 

5.7 

0.001  *** 

Average  quarterly  employment  (%) 

Report  sample 

59.7 

56.3 

3.4 

0.007  *** 

Eligible  sample 

61.0 

56.3 

4.7 

0.001  *** 

Fielded  sample 

62.0 

56.9 

5.1 

0.005  *** 

Respondent  sample 

65.6 

60.4 

5.2 

0.015  ** 

Employed  4 consecutive  quarters  (%) 

Report  sample 

23.6 

20.4 

3.2 

0.037  ** 

Eligible  sample 

24.1 

20.9 

3.2 

0.080  * 

Fielded  sample 

25.0 

21.7 

3.2 

0.164 

Respondent  sample 

26.0 

25.0 

1.1 

0.714 

Number  of  quarters  employed 

Report  sample 

9.5 

9.0 

0.5 

0.007  *** 

Eligible  sample 

9.8 

9.0 

0.8 

0.001  *** 

Fielded  sample 

9.9 

9.1 

0.8 

0.005  *** 

Respondent  sample 

10.5 

9.7 

0.8 

0.015  ** 

Average  annual  earnings  ($) 

Report  sample 

9,705 

8,852 

853 

0.007  *** 

Eligible  sample 

9,949 

8,959 

990 

0.009  *** 

Fielded  sample 

10,104 

9,097 

1,007 

0.032  ** 

Respondent  sample 

10,479 

9,577 

902 

0.104 

Ever  received  TANF  (%) 

Report  sample 

53.7 

55.8 

-2.1 

0.279 

Eligible  sample 

53.2 

55.5 

-2.4 

0.293 

Fielded  sample 

53.5 

55.0 

-1.4 

0.608 

Respondent  sample 

57.8 

59.4 

-1.6 

0.626 

Average  annual  TANF  received  ($) 

Report  sample 

934 

977 

-43 

0.362 

Eligible  sample 

931 

1,036 

-104 

0.063  * 

Fielded  sample 

904 

1,033 

-130 

0.059  * 

Respondent  sample 

1,036 

1,187 

-151 

0.082  * 

(continued) 
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Appendix  Table  D.51  (continued) 


Outcome 

ERA 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Average  annual  income  ($) 

Report  sample 

11,896 

11,117 

778 

0.012  ** 

Eligible  sample 

12,095 

11,296 

799 

0.030  ** 

Fielded  sample 

12,187 

11,468 

719 

0.117 

Respondent  sample 

12,885 

12,279 

606 

0.248 

SOURCE:  MDRC  calculations  from  the  state  administrative  records. 

NOTES:The  research  sample  includes  2,770  sample  members;  ERA  group:  1,627;  control  group:  1,143.  The 
eligible  sample  includes  1,957  sample  members;  ERA  group:  1,129;  control  group:  828.  The  fielded  sample 
includes  1,310  sample  members;  ERA  group:  757;  control  group:  553.  The  respondent  sample  includes  892 
sample  members;  ERA  group:  504;  control  group:  388. 

Statistical  significance  levels  are  indicated  as  follows:  ***  = 1 percent;  **  = 5 percent;  * = 10  percent. 
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Appendix  E 

Site-Specific  Publications  for  the 
Employment  Retention  and  Advancement  (ERA)  Project 
Tests  Discussed  in  This  Report 

The  following  site-specific  publications  on  ERA  are  referenced  in  this  report  and  can  be  found 
on  the  Web  sites  for  the  Administration  for  Children  and  Families  (ACF)  and  MDRC. 

ACF 

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/index.html 

MDRC 

http://www.mdrc.org/project_publications_14_9.html 
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Programs  Serving  Unemployed  TANF  Recipients  (Chapter  3) 


Texas 

The  Employment  Retention  and  Advancement  Project:  Results  from  the  Texas  ERA  Site. 
2006.  Karin  Martinson,  Richard  Hendra. 

Los  Angeles  EJC 

The  Employment  Retention  and  Advancement  Project:  A Comparison  of  Two  Job  Club 
Strategies:  The  Effects  of  Enhanced  Versus  Traditional  Job  Clubs  in  Los  Angeles. 
2008.  David  Navarro,  Gilda  Azurdia,  Gayle  Hamilton. 

Salem  (Oregon) 

The  Employment  Retention  and  Advancement  Project:  Results  from  the  Valuing 
Individual  Success  and  Increasing  Opportunities  Now  (VISION)  Program  in 
Salem,  Oregon.  2008.  Frieda  Molina,  Wan-Lae  Cheng,  Richard  Hendra. 


Programs  Serving  Employed  TANF  Recipients  (Chapter  4) 

Chicago 

The  Employment  Retention  and  Advancement  Project:  Results  from  the  Chicago  ERA 
Site.  2006.  Dan  Bloom,  Richard  Hendra,  Jocelyn  Page. 

Los  Angeles  RFS 

The  Employment  Retention  and  Advancement  Project:  Results  from  the  Los  Angeles 
Reach  for  Success  Program.  2009.  Jacquelyn  Anderson,  Stephen  Freedman,  Gayle 
Hamilton. 

Riverside  (California)  Phase  2 

The  Employment  Retention  and  Advancement  Project:  Results  from  Two  Education  and 
Training  Models  for  Employed  Welfare  Recipients  in  Riverside,  California.  2007. 
David  Navarro,  Stephen  Freedman,  Gayle  Hamilton. 

South  Carolina 

The  Employment  Retention  and  Advancement  Project:  Results  from  the  South  Carolina 
ERA  Site.  2005.  Susan  Scrivener,  Gilda  Azurdia,  Jocelyn  Page. 
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Programs  Serving  Employed  Non-TANF  Recipients  (Chapter  5) 


Cleveland 

The  Employment  Retention  and  Advancement  Project:  Findings  for  the  Cleveland 
ACHIEVE  Model:  Implementation  and  Early  Impacts  of  an  Employer-Based 
Approach  to  Encourage  Employment  Retention  Among  Low-Wage  Workers. 

2009.  Cynthia  Miller,  Vanessa  Martin,  Gayle  Hamilton  with  Lauren  Cates, 
Victoria  Deitch. 

Eugene  and  Medford  (Oregon) 

The  Employment  Retention  and  Advancement  Project:  Findings  for  the  Eugene  and 
Medford,  Oregon,  Models:  Implementation  and  Early  Impacts  for  Two  Programs 
That  Sought  to  Encourage  Advancement  among  Low-Income  Workers.  2009.  Frie- 
da Molina,  Mark  van  Dok,  Richard  Hendra,  Gayle  Hamilton,  Wan-Lae  Cheng. 

Riverside  (California)  PASS 

The  Employment  Retention  and  Advancement  Project:  Results  from  the  Post-Assistance 
Self-Sufficiency  (PASS)  Program  in  Riverside,  California.  2007.  David  Navarro, 
Mark  van  Dok,  Richard  Hendra. 


496 


References 


Acs,  Gregory,  and  Pamela  J.  Loprest.  2004.  Leaving  Welfare:  Employment  and  Well-Being  of 
Families  That  Left  Welfare  in  the  Post-Entitlement  Era.  Kalamazoo,  MI:  W.  E.  Upjohn  In- 
stitute for  Employment  and  Research. 

Anderson,  Jacquelyn,  Stephen  Freedman,  and  Gayle  Hamilton.  2009.  The  Employment  Reten- 
tion and  Advancement  Project:  Results  from  the  Los  Angeles  Reach  for  Success  Program. 
New  York:  MDRC. 

Andersson,  Fredrik,  Harry  J.  Holzer,  and  Julia  Lane.  2005.  Moving  Up  or  Moving  On:  Who 
Advances  in  the  Low-Wage  Labor  Market?  New  York:  Russell  Sage  Foundation  Publica- 
tions. 

Blank,  Rebecca  M.,  and  L.  Schmidt.  2001.  “Work,  Wages,  and  Welfare.”  In  Rebecca  M.  Blank 
and  Ron  Haskins  (eds.),  The  New  World  of  Welfare.  Washington,  DC:  Brookings  Institu- 
tion Press. 

Bloom,  Dan,  Richard  Hendra,  and  Jocelyn  Page.  2006.  The  Employment  Retention  and  Ad- 
vancement Project:  Results  from  the  Chicago  ERA  Site.  New  York:  MDRC. 

Bloom,  Howard  S.,  James  A.  Riccio,  and  Nandita  Verma  with  Johanna  Walter.  2005.  Promot- 
ing Work  in  Public  Housing:  the  Effectiveness  of  Jobs-Plus.  New  York:  MDRC. 

Bos,  Johannes  M.,  Susan  Scrivener,  Jason  Snipes,  and  Gayle  Hamilton  with  Christine  Schwartz  and 
Johanna  Walter.  2002.  Improving  Basic  Skills:  The  Effects  of  Adult  Education  in  Welfare-to- 
Work  Programs.  Washington,  DC:  U.S.  Department  of  Education  and  U.S.  Department  of 
Health  and  Human  Services. 

Economic  Policy  Institute.  2006.  “Wage  Growth  Slows  for  Most  Workers  Between  2000  and 
2005”  (January  11). 

Web  site:  http://www.epi.org/content.cfm/webfeatures_snapshots_200601 1 1. 

Freedman,  Stephen,  Jean  Tansey  Knab,  Lisa  A.  Gennetian,  and  David  Navarro.  2000.  The  Los 
Angeles  Jobs-First  GAIN  Evaluation:  Final  Report  on  a Work  First  Program  in  a Major 
Urban  Center.  New  York:  MDRC. 

Gennetian,  Lisa  A.,  Cynthia  Miller,  and  Jared  Smith.  2005.  Turning  Welfare  into  a Work 
Support:  Six-Year  Impacts  on  Parents  and  Children  from  the  Minnesota  Family  Investment 
Program.  New  York:  MDRC. 

Gottschalk,  Peter.  1982.  “Earnings  Mobility:  Permanent  Change  or  Transitory  Fluctuations?” 
Review  of  Economics  and  Statistics  (August):  450-456. 

Hamilton,  Gayle.  2002.  Moving  People  from  Welfare  to  Work:  Lessons  from  the  National 
Evaluation  of  Welfare-to-Work  Strategies.  Washington,  DC:  U.S.  Department  of  Health 
and  Human  Services  and  U.S.  Department  of  Education. 


497 


Holzer,  Harry  J.,  and  Karin  Martinson.  2005.  Can  We  Improve  Job  Retention  and  Advancement 
Among  Low-Income  Working  Parents?  Washington,  DC:  Urban  Institute. 

Huston,  Aletha  C.,  Cynthia  Miller,  Lashawn  Richburg-Hayes,  Greg  J.  Duncan,  Carolyn  A. 
Eldred,  Thomas  S.  Weisner,  Edward  Lowe,  Vonnie  A.  McLoyd,  Danielle  A.  Crosby,  Ma- 
rika N.  Ripke,  and  Cindy  Redcross.  2003.  New  Hope  for  Families  and  Children:  Five-Year 
Results  of  a Program  to  Reduce  Poverty  and  Reform  Welfare.  New  York:  MDRC. 

Johnson,  Rucker  C.,  and  Mary  E.  Corcoran.  2003.  “The  Road  to  Economic  Self-Sufficiency: 
Job  Quality  and  Job  Transition  Patterns  After  Welfare  Reform.”  Journal  of  Public  Policy 
Analysis  and  Management  22,  4:  625. 

Komfeld,  Robert,  and  Howard  S.  Bloom.  1999.  “Measuring  Program  Impacts  on  Earnings  and 
Employment:  Do  Unemployment  Insurance  Wage  Reports  from  Employers  Agree  with 
Surveys  of  Individuals?”  Journal  of  Labor  Economics  17,  1:  168-197. 

Lerman,  Robert.  2005.  “Single  Parents’  Earning  Monitor.”  Number  3.  Washington,  DC:  Urban 
Institute. 

Martinson,  Karin,  and  Richard  Hendra.  2006.  The  Employment  Retention  and  Advancement 
Project:  Results  from  the  Texas  ERA  Site.  New  York:  MDRC. 

Michalopoulos,  Charles.  2005.  Does  Making  Work  Pay  Still  Pay?  An  Update  on  the  Effects  of 
Four  Earnings  Supplement  Programs  on  Employment,  Earnings,  and  Income.  New  York: 
MDRC. 

Michalopoulos,  Charles,  Doug  Tattrie,  Cynthia  Miller,  Philip  K.  Robins,  Pamela  Morris,  David 
Gyarmati,  Cindy  Redcross,  Kelly  Foley,  and  Reuben  Ford.  2002.  Making  Work  Pay:  Final 
Report  on  the  Self-Sufficiency  Project  for  Long-Term  Welfare  Recipients.  Ottawa:  Social 
Research  and  Demonstration  Corporation. 

Miller,  Cynthia,  Victoria  Deitch,  and  Aaron  Hill.  2010.  Paths  to  Advancement  for  Single 
Parents  in  the  Employment  Retention  and  Advancement  (ERA)  Project.  New  York: 
MDRC. 

Mishel,  Lawrence  (ed.),  Jared  Bernstein,  and  Sylvia  Allegretto.  2007.  The  State  of  Working 
America.  Ithaca,  NY : Cornell  University  Press. 

Molina,  Frieda,  Wan-Lae  Cheng,  and  Richard  Hendra.  2008.  The  Employment  Retention  and 
Advancement  Project:  Results  from  the  Valuing  Individual  Success  and  Increasing  Oppor- 
tunities Now  (VISION)  Program  in  Salem,  Oregon.  New  York:  MDRC. 

Navarro,  David,  Gilda  Azurdia,  and  Gayle  Hamilton.  2008.  The  Employment  Retention  and 
Advancement  Project:  A Comparison  of  Two  Job  Club  Strategies:  The  Effects  of  Enhanced 
Versus  Traditional  Job  Clubs  in  Los  Angeles.  New  York:  MDRC. 

Mumane,  Richard  J.,  John  B.  Willett,  and  Kathryn  Parker  Boudett.  1995.  “Do  High  School 
Dropouts  Benefit  from  Obtaining  a GED?”  Educational  Evaluation  and  Policy  Analysis 
17,  2 (Summer):  133-147. 


498 


Rangarajan,  Anu,  and  Tim  Novak.  1999.  The  Struggle  to  Sustain  Employment:  The  Effective- 
ness of  the  Postemployment  Sendees  Demonstration.  Princeton,  NJ:  Mathematica  Policy 
Research,  Inc. 

Redcross,  Cindy,  Victoria  Deitch,  and  Mary  Farrell.  2009.  Benefit-Cost  Findings  for  Three 
Programs  in  the  Employment  Retention  and  Advancement  (ERA)  Project.  New  York: 
MDRC. 

Riccio,  James  A.,  Helen  Bewley,  Verity  Campbell-Barr,  Richard  Dorsett,  Gayle  Hamilton, 
Lesley  Hoggart,  Alan  Marsh,  Cynthia  Miller,  Kathryn  Ray,  and  Sandra  Vegeris.  2008.  Im- 
plementation and  Second-Year  Impacts  for  Lone  Parents  in  the  UK  Employment  Retention 
and  Advancement  (ERA)  Demonstration.  Department  for  Work  and  Pensions  Research  Re- 
port No.  489.  Leeds,  UK:  Corporate  Document  Services. 

Schochet,  Peter,  and  Ann  Rangarajan.  2004.  Characteristics  of  Low-Wage  Workers  and  Their 
Labor  Market  Experiences:  Evidence  from  the  Mid-  to  Late  1990s.  Washington,  DC:  U.S. 
Department  of  Health  and  Human  Services. 

Smith,  Ralph  E.,  and  Vavrichek,  Bruce.  1992.  “The  Mobility  of  Minimum  Wage  Workers.” 
Industrial  and  Labor  Relations  Review  46  (October):  82-88. 

Stone,  Robyn  I.,  with  Joshua  M.  Weiner.  2001.  Who  Will  Care  for  Us?  Addressing  the  Long- 
Term  Care  Workforce  Crisis.  Washington,  DC:  Urban  Institute  and  American  Association 
of  Homes  and  Services  for  the  Aging. 

Swaim,  Paul.  1997.  “Earnings  Inequality,  Low-Paid  Employment  and  Earnings  Mobility.” 
Employment  Outlook  2.  Paris:  Organization  for  Economic  Cooperation  and  Development. 

Theodos,  Brett,  and  Robert  Bednarzik.  2006.  “Earnings  Mobility  and  Low-Wage  Workers  in  the 
United  States.”  BLS  Monthly  Labor  Review  Online. 

Web  site:  http://www.bls.gov/opub/mlr/. 

U.S.  Census  Bureau.  2005.  “Current  Population  Survey,  March  2005.” 

Web  site:  http://www.census.gov/cps/. 

U.S.  Department  of  Labor,  Bureau  of  Labor  Statistics.  2008.  “May  2008  National  Occupational 
Employment  and  Wage  Estimates.”  Web  site:  http://www.bls.gov/oes/current/oes_nat.htm. 


499 


MDRC  Publications  on  the  Employment 
Retention  and  Advancement  Project 


The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Los  Angeles  Reach  for  Success  Program. 

2009.  Jacquelyn  Anderson,  Stephen  Freedman,  Gayle  Hamilton. 

The  Employment  Retention  and  Advancement  Project: 

Findings  for  the  Eugene  and  Medford,  Oregon,  Models:  Implementation  and  Early  Impacts  for 
Two  Programs  That  Sought  to  Encourage  Advancement  Among  Low-Income  Workers. 

2009.  Frieda  Molina,  Mark  van  Dok,  Richard  Hendra,  Gayle  Hamilton,  Wan-Lae  Cheng. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Substance  Abuse  Case  Management  Program  in  New  York  City 
2009.  John  Martinez,  Gilda  Azurdia,  Dan  Bloom,  and  Cynthia  Miller. 

Findings  for  the  Cleveland  Achieve  Model: 

Implementation  and  Early  Impacts  of  an  Employer-Based  Approach  to  Encourage  Employment 
Retention  Among  Low-  Wage  Workers. 

2008.  Cynthia  Miller,  Vanessa  Martin,  Gayle  Hamilton  with  Lauren  Cates,  Victoria  Deitch. 

A Comparison  of  Two  Job  Club  Strategies: 

The  Effects  of  Enhanced  Versus  Traditional  Job  Clubs  in  Los  Angeles. 

2008.  David  Navarro,  Gilda  Azurdia,  Gayle  Hamilton. 

The  Employment  Retention  and  Advancement  Project: 

Impacts  for  Portland’s  Career  Builders  Program. 

2008.  Gilda  Azurdia,  Zakia  Barnes. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Valuing  Individual  Success  and  Increasing  Opportunities  Now  (VISION) 
Program  in  Salem,  Oregon. 

2008.  Frieda  Molina,  Wan-Lae  Cheng,  Richard  Hendra. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  Two  Education  and  Training  Models  for  Employed  Welfare  Recipients  in 
Riverside,  California. 

2007.  David  Navarro,  Stephen  Freedman,  Gayle  Hamilton. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Personal  Roads  to  Individual  Development  and  Employment  (PRIDE) 
Program  in  New  York  City. 

2007.  Dan  Bloom,  Cynthia  Miller,  Gilda  Azurdia. 


501 


The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Post-Assistance  Self-Sufficiency  (PASS)  Program  in  Riverside,  California. 
2007.  David  Navarro,  Mark  van  Dok,  Richard  Hendra. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  Minnesota ’s  Tier  2 Program. 

2007.  Allen  LeBlanc,  Cynthia  Miller,  Karin  Martinson,  Gilda  Azurdia. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Chicago  ERA  Site. 

2006.  Dan  Bloom,  Richard  Hendra,  Jocelyn  Page. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  the  Texas  ERA  Site. 

2006.  Karin  Martinson,  Richard  Hendra. 

The  Employment  Retention  and  Advancement  Project: 

Results  from  the  South  Carolina  ERA  Site. 

2005.  Susan  Scrivener,  Gilda  Azurdia,  Jocelyn  Page. 

The  Employment  Retention  and  Advancement  Project: 

Early  Results  from  Four  Sites. 

2005.  Dan  Bloom,  Richard  Hendra,  Karin  Martinson,  Susan  Scrivener. 

Service  Delivery  and  Institutional  Linkages: 

Early  Implementation  Experiences  of  Employment  Retention  and  Advancement  Programs. 
2003.  Jacquelyn  Anderson,  Karin  Martinson. 

New  Strategies  to  Promote  Stable  Employment  and  Career  Progression: 

An  Introduction  to  the  Employment  Retention  and  Advancement  Project. 

U.S.  Department  of  Health  and  Human  Services. 

2002.  Dan  Bloom,  Jacquelyn  Anderson,  Melissa  Wavelet,  Karen  N.  Gardiner,  Michael  E. 
Fishman. 


NOTE:  A complete  publications  list  is  available  from  MDRC  and  on  its  Web  site 
( www.mdrc.org),  from  which  copies  of  reports  can  also  be  downloaded. 


502 


